Systems for streaming and video conferencing

Abstract

To provide means that enables discussions among multiple people who watch a stream-distributed content.SOLUTION: When a user enters a specified URL in a Web browser, a login page (Fig. 3 (A)) is displayed. When the user successfully logs in, the Web browser displays a stream-distributed distribution page (Fig. 3 (B)) that displays a content stream-distributed from a distribution source device via a server device. In a case in which a "Video Conference" button is displayed on the streaming distribution page (Fig. 3 (C)), when the user presses the "Video Conference" button, the Web browser displays a video conference room page (Fig. 3 (D)) for conducting a video conference between users in a predetermined same group among a plurality of users who are watching the same content.SELECTED DRAWING: Figure 3

Description

Technical Field

[0001] The present invention relates to a system for assisting smooth communication among people through data communication.

Background Art

[0002] A system (hereinafter referred to as a "streaming distribution system") that distributes content such as video in one direction from a communication device of a distribution source to communication devices of a large number of destinations via a communication network such as the Internet, and plays back the content received sequentially at the communication devices of the destinations, has become widespread. According to the streaming distribution system, a viewer who is a user of the communication device at the destination can view the content distributed from the distribution source.

[0003] In addition, a system (hereinafter referred to as a "video conference system") in which each of a large number of communication devices transmits and receives surrounding sounds and images substantially in real time via a communication network such as the Internet has become widespread. According to the video conference system, participants in a conference who are users of the communication devices can talk to each other while seeing the faces of other participants who are in a distant location.

[0004] In recent years, a system (hereinafter referred to as a "webinar system") that integrates a streaming distribution system and a video conference system and streams the state of a conference conducted by people called panelists to the communication devices used by viewers in audio and video has been increasingly widespread. As a patent document disclosing a technique related to the webinar system, for example, there is Patent Document 1. Patent Document 1 describes a mechanism in which among viewers who have received a streaming distribution service, those who wish (participants) can talk to the distributor when they receive permission, and the conversation between the distributor and the participants is streamed. According to the mechanism described in Patent Document 1, if a viewer of the streaming distribution wishes, they can participate as a panelist or the like in the content of the streaming distribution as a participant.

Prior Art Documents

Patent Documents

[0005] [Patent Document 1] Japanese Patent Publication No. 2018-101965 [Overview of the project] [Problems that the invention aims to solve]

[0006] It is desirable for multiple people who have watched lectures, videos, or other content to discuss the content they have watched, as this will deepen their understanding of the content. Furthermore, when multiple people discuss a particular topic, it is desirable for them to watch content related to that topic beforehand, as this content can serve as a starting point for a lively discussion.

[0007] When multiple people gather together, after they have viewed content such as lectures or videos projected using a projector, those sitting near each other can form groups and discuss the content they have viewed.

[0008] However, when people are individually in separate locations and watching streamed content, they cannot discuss that content with each other.

[0009] In view of the circumstances described above, the present invention aims to provide a means that enables discussion among multiple people who are watching content being streamed. [Means for solving the problem]

[0010] To solve the above problems, the present invention provides, in a first embodiment, a system comprising: a streaming distribution means that receives audiovisual data from a source distribution device and streams the audiovisual data to a plurality of destination distribution devices; and a video conferencing relay means that receives audiovisual data from each of the plurality of destination distribution devices that are participating in a video conference, including images captured by cameras connected to the destination devices and sound picked up by microphones connected to the destination devices, synthesizes the received audiovisual data to generate audiovisual data for video conferencing, and transmits the generated audiovisual data for video conferencing to each of the two or more destination distribution devices participating in the video conference, thereby relaying a video conferencing.

[0011] According to the system of the first embodiment, multiple people who are watching the streamed content can have a discussion.

[0012] In the system according to the first embodiment, a second embodiment may be adopted in which the streaming distribution means does not distribute audiovisual data to a destination device that is receiving audiovisual data from the video conferencing relay means.

[0013] According to the system of the second embodiment, if the people participating in the video conference do not need to view the content streamed from the source device, it is desirable that no unnecessary data communication occurs.

[0014] In the system according to the second embodiment, a third embodiment may be adopted in which the streaming distribution means resumes distribution of audiovisual data to a distribution destination device that has stopped receiving audiovisual data from the video conferencing relay means.

[0015] According to the system of the third aspect, a person who has left a video conference can view the content being streamed from the source device.

[0016] In the system according to the first embodiment, a fourth embodiment may be adopted in which the streaming distribution means distributes audiovisual data to a destination device that is receiving audiovisual data from the video conferencing relay means at a lower bitrate than to a destination device that is not receiving audiovisual data from the video conferencing relay means.

[0017] According to the system of the fourth embodiment, while a video conference is taking place, the strain on communication bandwidth caused by streaming content from the source device is reduced.

[0018] In a system according to any of the first to third embodiments, a fifth embodiment may be adopted in which the video conferencing relay means receives audiovisual data from the source device and generates audiovisual data for the video conference by combining the audiovisual data received from the source device with the audiovisual data received from each of the two or more destination devices participating in the video conference.

[0019] According to the system of the fifth aspect, participants in a video conference can view content streamed from the source device while conducting the video conference.

[0020] A sixth embodiment may be adopted in a system according to any of the first to fifth embodiments, wherein the system includes a grouping means that generates a plurality of groups by grouping a plurality of destination devices that receive audiovisual data from the streaming distribution means, and the video conferencing relay means transmits to each of the plurality of destination devices, with respect to each of the plurality of groups generated by the grouping of the grouping means, two or more destination devices belonging to the group that participate in the video conference, audiovisual data for the video conference generated by synthesizing the audiovisual data received from each of the two or more destination devices.

[0021] According to the system of the sixth aspect, multiple people watching streaming content can hold discussions in groups.

[0022] In the system according to the sixth aspect, the grouping means may adopt, as a seventh aspect, a configuration in which, for each of a plurality of destination devices that receive audiovisual data from the streaming distribution means, attribute data indicating an attribute of the destination device or a user of the destination device is acquired, and grouping is performed based on the attribute data.

[0023] According to the system according to the seventh aspect, conversations can be held among people in a desirable combination.

[0024] In the system according to the seventh aspect, an attribute data receiving means for receiving, as attribute data, data input by a user of the destination device from each of a plurality of destination devices that receive audiovisual data from the streaming distribution means may be provided, and the grouping means may adopt, as an eighth aspect, a configuration in which grouping is performed based on the attribute data received by the attribute data receiving means.

[0025] According to the system according to the eighth aspect, conversations can be held among people in a desirable combination according to data input by people during viewing of content being stream-distributed.

[0026] In the system according to any one of the sixth to eighth aspects, the grouping means may adopt, as a ninth aspect, a configuration in which relational data indicating a relationship between two or more destination devices is acquired, and grouping is performed based on the relational data.

[0027] According to the system according to the ninth aspect, a plurality of people having a specific relationship can be made to talk in the same group. Also, according to the system according to the ninth aspect, a plurality of people having a specific relationship can be made not to talk in the same group.

[0028] In a system according to any of the sixth to ninth embodiments, the grouping means further groups multiple destination devices belonging to each of the multiple groups generated by the grouping to generate multiple subgroups, and the video conferencing relay means transmits to each of the two or more destination devices, with respect to each of the multiple subgroups generated by the grouping of the grouping means, audiovisual data for video conferencing generated by synthesizing audiovisual data received from each of the two or more destination devices belonging to the subgroup. This configuration may be adopted as the tenth embodiment.

[0029] According to the system of the tenth aspect, the people who have had a discussion can divide into smaller groups and continue their discussions.

[0030] An eleventh embodiment may be adopted in which, in the system according to the first embodiment, the streaming distribution means transmits display instruction data to each of a plurality of destination devices that instructs the display of a web page containing images represented by audiovisual data and images of operators for participating in a video conference, and the video conference relay means transmits display instruction data to the destination device that instructs the display of a web page containing images represented by audiovisual data for video conferencing when an operation is performed on the operator for participating in the video conference.

[0031] According to the system of the 11th embodiment, people can use a data communication device capable of running a web browser as a destination device to view content streamed from a source device and participate in a video conference by performing operations on controls displayed with that content.

[0032] In the system according to the 11th embodiment, a configuration may be adopted as the 12th embodiment in which the display instruction data transmitted by the streaming distribution means instructs the termination of the display of the web page being displayed in accordance with the display instruction data when an operation is performed on the operator for participating in the video conference.

[0033] According to the system of the 12th embodiment, it is desirable that no unnecessary information is displayed when people participating in a video conference do not need to view the content streamed from the source device.

[0034] In the system according to the 12th embodiment, a 13th embodiment may be adopted in which the display instruction data transmitted by the video conferencing relay means instructs the display of a web page containing an image of an operator for instructing to end participation in a video conference, and when an operation is performed on the operator, the display of the web page is instructed to end, and when an operation is performed on the operator for instructing to end participation in a video conference, the streaming distribution means transmits display instruction data to the destination device instructing the display of a web page containing an image of an operator for joining a video conference and an image of an audiovisual data received from the distribution source device.

[0035] According to the system of the 13th embodiment, people can exit a video conference by performing operations on controls displayed along with the video conference images, and can also view content streamed from the source device.

[0036] The present invention relates to a distribution destination device for receiving audiovisual data transmitted by a system according to a first embodiment, comprising: receiving means for receiving audiovisual data from the system; transmitting means for transmitting audiovisual data representing images captured by a connected camera and sounds picked up by a connected microphone to the system; display instruction means for causing the image represented by the audiovisual data received by the receiving means to be displayed on a display device; sound instruction means for causing the sound represented by the audiovisual data received by the receiving means to be pronounced by a sound-producing device; and operation receiving means for receiving operations on the image displayed by the display device in accordance with the display instruction means, wherein the display instruction means is received by the receiving means from the streaming distribution means of the system A fourteenth embodiment of a distribution destination device is provided, which, when displaying an image represented by audiovisual data on the display device, displays an operator for joining a video conference on the display device, and when the operation receiving means receives an operation for the operator for joining a video conference, the transmission means starts transmitting audiovisual data to the system, the receiving means starts receiving audiovisual data from the video conference relay means of the system, the display instruction means displays the image represented by the audiovisual data received by the receiving means from the video conference relay means on the display device, and the sound instruction means causes the sounding device to sound the sound represented by the audiovisual data received by the receiving means from the video conference relay means.

[0037] According to the 14th embodiment of the receiving device, people can participate in a video conference by viewing content streamed from the source device and performing operations on controls displayed with that content.

[0038] In the distribution destination device according to the 14th embodiment, if the operation receiving means receives an operation for an operator for joining the video conference, the receiving means may terminate the reception of audiovisual data from the streaming distribution means of the system, which may be adopted as the 15th embodiment.

[0039] According to the distribution destination device of the 15th embodiment, it is desirable that no unnecessary data communication occurs when people participating in a video conference do not need to view the content streamed from the distribution source device.

[0040] In a distribution destination device according to the 15th embodiment, the 16th embodiment may be configured such that, when the receiving means causes the display device to display an image represented by audiovisual data received from the video conferencing relay means of the system, the display instruction means causes the display device to display an operator for instructing the termination of participation in a video conference, and when the operation receiving means receives an operation on the operator for instructing the termination of participation in a video conference, the transmitting means terminates the transmission of audiovisual data to the system, the receiving means receives audiovisual data from the streaming distribution means of the system in place of the audiovisual data from the video conferencing relay means of the system, the display instruction means causes the display device to display an image represented by the audiovisual data received by the receiving means from the streaming distribution means, and the sound instruction means causes the sound device to sound the sound represented by the audiovisual data received by the receiving means from the streaming distribution means.

[0041] According to the 16th embodiment of the receiving device, people can exit a video conference by performing operations on controls displayed along with the video conference image, and at the same time, view content streamed from the distributing device.

[0042] The present invention provides, in its 17th aspect, a program for causing a computer to function as the receiving means, transmitting means, display instruction means, sound output instruction means, and operation reception means of a distribution destination device according to its 14th aspect.

[0043] According to the program of the 17th embodiment, people can use a computer to view content streamed from a distribution device and participate in a video conference by performing operations on controls displayed with that content. [Brief explanation of the drawing]

[0044] [Figure 1] A diagram showing the configuration of a system according to one embodiment. [Figure 2] A diagram showing the data structure of a user information table stored in a server device according to one embodiment. [Figure 3] A schematic diagram showing the screen displayed on the screen of a display device connected to a distribution destination device according to one embodiment. [Figure 4] A schematic diagram showing the screen displayed on the display device connected to the destination device according to one modified example. [Figure 5] A schematic diagram showing the screens displayed on the display devices connected to the source and destination devices according to one modified example. [Figure 6] A schematic diagram showing the screen displayed on the display device connected to the destination device according to one modified example. [Figure 7] A schematic diagram showing the screen displayed on the display device connected to the source device according to one modified example. [Figure 8] A schematic diagram showing the screen displayed on the display device connected to the destination device according to one modified example. [Figure 9] A schematic diagram showing the screen displayed on the display device connected to the destination device according to one modified example. [Figure 10] This diagram illustrates the relationship between the time period during which the organizers stream content in real time and the time period during which participating users view that content, in one modified example. [Figure 11] A schematic diagram showing the screen displayed on the display device connected to the destination device according to one modified example. [Figure 12] A schematic diagram showing the screen displayed on the display device connected to the destination device according to one modified example. [Figure 13]A schematic diagram showing the screen displayed on the display device connected to the destination device according to one modified example. [Figure 14] A schematic diagram showing the screen displayed on the display device connected to the destination device according to one modified example. [Figure 15] A schematic diagram showing the screen displayed on the display device connected to the destination device according to one modified example. [Figure 16] A schematic diagram showing the screen displayed on the display device connected to the destination device according to one modified example. [Figure 17] A schematic diagram showing the screen displayed on the display device connected to the destination device according to one modified example. [Figure 18] A schematic diagram showing the screen displayed on the display device connected to the destination device according to one modified example. [Figure 19] A schematic diagram showing the screen displayed on the display device connected to the destination device according to one modified example. [Figure 20] A schematic diagram showing the screen displayed on the display device connected to the destination device according to one modified example. [Figure 21] A schematic diagram showing the screen displayed on the display device connected to the destination device according to one modified example. [Figure 22] A schematic diagram showing the screen displayed on the display device connected to the source device according to one modified example. [Figure 23] A schematic diagram showing the screen displayed on the display device connected to the source device according to one modified example. [Figure 24] A schematic diagram showing the screen displayed on the display device connected to the destination device according to one modified example. [Figure 25] A schematic diagram showing the screen displayed on the display device connected to the destination device according to one modified example. [Figure 26]A schematic diagram showing the screen displayed on the display device connected to the source device according to one modified example. [Figure 27] A schematic diagram showing the screen displayed on the display device connected to the source device according to one modified example. [Figure 28] A schematic diagram showing the screen displayed on the display device connected to the destination device according to one modified example. [Modes for carrying out the invention]

[0045] The following describes a system 10 according to one embodiment of the present invention, and the source device 11 and destination devices 12(1), 12(2), ..., 12(n) (where n is a natural number of 2 or greater) used by the user of system 10. Hereinafter, when destination devices 12(1), 12(2), ..., 12(n) are not distinguished from each other, they will be collectively referred to as destination device 12.

[0046] Figure 1 shows the configuration of system 10. However, in Figure 1, in addition to system 10, the source device 11 and the destination device 12 are also shown. System 10 comprises server devices 101, 102, and 103. Server devices 101, 102, 103, the source device 11, and the destination device 12 are connected to the communication network 9. Server devices 101, 102, and 103 communicate data with other devices connected to the communication network 9.

[0047] Server device 101 is a server device that plays the role of streaming audiovisual data transmitted from source device 11 to each of the destination devices 12. Hereinafter, the audiovisual data distributed from source device 11 to each of the destination devices 12 via server device 101 will be referred to as streaming audiovisual data.

[0048] Audiovisual data for streaming distribution includes, for example, videos with sound that are pre-recorded and played back during streaming distribution, and videos with sound that include images captured by a camera connected to the streaming source device 11 during streaming distribution and audio picked up by a microphone connected to the streaming source device 11 during streaming distribution.

[0049] The server device 102 is a server device that synthesizes audiovisual data transmitted from the source device 11 and the destination device 12 to generate audiovisual data for video conferencing (hereinafter referred to as "audiovisual data for video conferencing"), and transmits the generated audiovisual data for video conferencing to each of the destination devices 12, thereby enabling users of the destination devices 12 to conduct video conferences.

[0050] The audiovisual data that server device 102 receives from each of the destination devices 12 is video with sound, including images captured by cameras connected to destination devices 12 during the video conference and audio picked up by microphones connected to destination devices 12 during the video conference. The audiovisual data that server device 102 receives from source device 11 is audiovisual data for streaming distribution.

[0051] Server device 103 is a server device that manages the users of the source device 11 and the destination device 12. Specifically, server device 103 performs user authentication, grouping of users participating in video conferencing, and data collection from users.

[0052] The server device 103 stores a data table (hereinafter referred to as the user information table) with the configuration shown in Figure 2 in order to manage users. The user information table is a collection of data records for each user who uses the system 10, and each record has the following items: "User ID", "Password", "Source / Destination", "Status", "Attribute", "Relationship", "Group Number", and "Subgroup Number".

[0053] The "User ID" and "Password" fields store the username and password, which are used to authenticate and identify the user. The "Source / Destination" field stores information indicating whether the user is the source or destination of the audiovisual data for streaming. The "Status" field stores information indicating whether the user is offline, receiving a streaming broadcast, or participating in a video conference.

[0054] The "Attributes" field stores information indicating the attributes of the user or the device used by the user (the source device 11 or the destination device 12). The "Attributes" field has multiple sub-items, namely the "Attribute 1" field, the "Attribute 2" field, ..., and the "Attribute m" field (where m is a natural number), so that it can store information indicating multiple types of attributes.

[0055] Examples of user attributes indicated by the information stored in the "Attributes" field include, but are not limited to, the user's age, the user's current location, the user's personality, the user's preferences, and the user's opinions on a particular topic. Similarly, examples of device attributes (source device 11 or destination device 12) indicated by the information stored in the "Attributes" field include, but are not limited to, the screen size of the display device connected to the device, the device's data communication method (e.g., connected to a mobile network, wired connection, etc.), and the device's data processing speed.

[0056] The "Relationship" column stores information (relationship data) indicating the relationship between specific users. For example, a video conference may be conducted in Japanese, and user B, an interpreter, may be supporting user A, who does not speak Japanese. In such a case, user A and user B must always be in the same group during the video conference. In this case, user A's "Relationship" column would store information such as "Same group as user B," and user B's "Relationship" column would store information such as "Same group as user A."

[0057] Furthermore, if user X and user Y are family members, there may be cases where you want to prevent them from being in the same group during a video conference. In such cases, the "Relationship" field for user X will store information such as "In a different group from user Y," and the "Relationship" field for user Y will store information such as "In a different group from user X."

[0058] The "Group Number" field stores information identifying the group to which the user belongs. In video conferences, multiple users belonging to one group may further divide into smaller groups for discussion. The "Subgroup Number" field stores information identifying the subgroup to which the user belongs.

[0059] For example, if a user of the source device 11 enters grouping conditions using the source device 11, the server device 103 groups the users of the destination device 12 to satisfy those conditions and the conditions indicated by the information stored in the "Relationship" column of the user information table, thereby generating multiple groups. The server device 103 then updates the information in the "Group Number" column and "Subgroup Number" column of the user information table according to the grouping results.

[0060] Examples of conditions for grouping include, but are not limited to, the acceptable number of users belonging to each group (or subgroup), and the combination of user attributes belonging to each group (for example, prioritizing the inclusion of users from different regions in the same group, prioritizing the inclusion of users with similar characteristics in different groups, etc.).

[0061] For example, if a user of the source device 11 issues an instruction to conduct a survey using the source device 11, the server device 103 sends data to each of the destination devices 12 that prompts the users of the destination devices 12 to enter answers to pre-prepared questions on a web page (hereinafter referred to as the survey page), and receives data from the destination devices 12 indicating the answers entered by the users on the survey page. The server device 103 stores the data indicating the answers received from the destination devices 12 in the "Attributes" column of the user information table. Therefore, it becomes possible to group users for subsequent video conferences according to the information entered by the users of the destination devices 12.

[0062] Server devices 101, 102, and 103 are each composed of a computer equipped with a processor, memory, and a communication interface.

[0063] The server device 101 functions as a device equipped with streaming distribution means that receives audiovisual data for streaming distribution from the source device 11 by having the computer's processor perform data processing according to a program stored in memory, and streams the audiovisual data for streaming distribution to each of the multiple destination devices 12.

[0064] The server device 102 functions as a device equipped with a video conferencing relay means that relays a video conference by having the computer's processor perform data processing according to a program stored in memory, receiving audiovisual data from each of two or more destination devices 12 that are participating in a video conference among multiple destination devices 12 that are receiving audiovisual data for streaming distribution from the streaming distribution means of the server device 101, synthesizing the received audiovisual data to generate audiovisual data for video conferencing, and transmitting the generated audiovisual data for video conferencing to each of the two or more destination devices 12 participating in the video conference.

[0065] The server device 103 functions as a device comprising a grouping means that generates multiple groups by grouping multiple destination devices 12 that receive audiovisual data for streaming distribution from the streaming distribution means of the server device 101, by having the computer's processor perform data processing according to a program stored in memory, and an attribute data receiving means that receives data input by the user of each of the multiple destination devices 12 that receive audiovisual data for streaming distribution from the streaming distribution means of the server device 101 as attribute data.

[0066] The source device 11 is a terminal device used by the user who is the source of the audiovisual data for streaming distribution. The source device 11 transmits to the server device 101 as audiovisual data for streaming distribution, including data representing videos with sound or still images that have been stored in advance, data representing videos with sound that include images taken by a camera and sound picked up by a microphone during streaming distribution, or data representing videos with sound obtained by synthesizing such videos with sound.

[0067] The destination device 12 is a terminal device used by the user to whom the audiovisual data for streaming is delivered. The destination device 12 receives the audiovisual data for streaming transmitted from the source device 11 via the server device 101 and plays the video with sound represented by the received audiovisual data for streaming.

[0068] Users of the receiving device 12 can also participate in the video conference. The receiving device 12 transmits audiovisual data representing a video with sound, including images captured by the camera and audio picked up by the microphone during the video conference, to the server device 102. It also receives audiovisual data for video conferencing transmitted from the server device 102 and plays the video with sound represented by the received audiovisual data for video conferencing.

[0069] The user of the receiving device 12, for example, while watching a video with sound represented by audiovisual data for streaming distributed from the source device 11 via the server device 101, displays a survey page according to data transmitted from the server device 103, and sends data indicating the answers entered by the user to the survey page back to the server device 103.

[0070] Each of the source device 11 and the destination device 12 consists of a computer equipped with a processor, memory, and a communication interface. The computer comprising the source device 11 or the destination device 12 is connected to a camera, microphone, display device (such as an LCD display), sound-producing device (speaker), and operation receiving device (such as a keyboard, mouse, or touch panel). The computer comprising the source device 11 or the destination device 12 may also have at least one of the camera, microphone, display device, sound-producing device, and operation receiving device built-in.

[0071] The source device 11 functions as a device equipped with a transmission means that transmits audiovisual data representing a video with sound, including images captured by a camera and sound picked up by a microphone, to the server device 101 by having the computer's processor perform data processing according to a program stored in memory.

[0072] The destination device 12 functions as a device comprising the following components by having the computer's processor perform data processing according to a program stored in memory. A receiving means that receives audiovisual data for streaming distribution from server device 101, audiovisual data for video conferencing from server device 102, and data instructing the display of a survey page from server device 103. A transmission means that transmits audiovisual data representing images captured by a connected camera and audio picked up by a connected microphone to a server device 102, and transmits data representing the user's answers to the survey page to a server device 103. Display instruction means for displaying an image or questionnaire page represented by streaming audiovisual data or video conferencing audiovisual data received by the receiving means on a display device. A pronunciation instruction means that causes a pronunciation device to pronounce the sound represented by the audiovisual data for streaming distribution or audiovisual data for video conferencing received by the receiving means. An operation receiving means that accepts user operations on an image displayed by a display device in accordance with a display instruction means.

[0073] Figure 3 schematically shows the screen displayed on the display device connected to the destination device 12 when a user of the destination device 12 views content streamed using system 10 or participates in a video conference.

[0074] The user of the destination device 12 performs a predetermined operation on the destination device 12 to launch a web browser, and then enters a predetermined URL into the web browser. In response to the input of that URL, the destination device 12 displays a web page (hereinafter referred to as the login page) as shown in Figure 3(A) on the display device, according to the display instruction data (hereinafter referred to as the display instruction data) sent from the server device 103. The display instruction data is data written in a language such as HTML that is interpreted by the web browser.

[0075] On the login page, the user enters their user ID and password and performs an action (for example, clicking with the mouse cursor or touching the screen) to click the "OK" button. In response to this action, the user ID and password are sent from the destination device 12 to the server device 103. The server device 103 searches the user information table for the user ID and password combination received from the destination device 12 and authenticates and identifies the user. If the user enters the correct user ID and password, the server device 103 changes the information in the "Status" column of the user's data record in the user information table from offline to receiving streaming distribution, and then redirects the communication connection from the destination device 12 to a communication destination identified by a predetermined URL. The server device 103 also sends instruction data to the destination device 12 instructing it to close the login page. In response to this instruction data, the login page displayed on the screen of the display device connected to the destination device 12 is closed.

[0076] As a result of the redirection described above, the destination device 12 establishes a communication connection with the server device 101. The server device 101 sends display instruction data to the destination device 12 instructing it to display a web page (hereinafter referred to as the streaming page) for playing the audiovisual data for streaming that has been sent from the source device 11. The destination device 12 displays the streaming page shown in Figure 3(B) on its display device in accordance with the display instruction data received from the server device 101. In the example in Figure 3(B), an image representing the situation in which speaker K is giving a lecture while displaying presentation materials on screen S is displayed in area A1.

[0077] While the streaming page is displayed, the receiving device 12 causes the audio playback device (speaker) to play the audio represented by the audiovisual data for streaming. The user can view the content distributed from the source device 11 via the server device 101 by viewing the images displayed on the streaming page and listening to the audio played in conjunction with the display of the streaming page.

[0078] Area A1 of the streaming page displays images represented by audiovisual data for streaming. Area A2 of the streaming page displays virtual controls that accept user input. The controls displayed in Area A2 change, for example, in response to user input on the source device 11. In the streaming page illustrated in Figure 3(B), Area A2 contains only an "End" button for receiving an instruction to terminate use of System 10.

[0079] For example, when the first part of speaker K's lecture ends, the user of the source device 11 performs a predetermined operation to instruct the source device 11 to switch to a video conference. In response to this operation, the source device 11 sends predetermined instruction data to the server device 101. Upon receiving this instruction data, the server device 101 sends display instruction data to the destination device 12 instructing it to display a streaming page with an added "Go to Video Conference" button (an example of an operator for joining a video conference) in area A2. As a result, the screen of the display device connected to the destination device 12 displays a streaming page as shown in Figure 3(C).

[0080] In addition, in area A1 of the streaming page illustrated in Figure 3(C), content distributed from the source device 11 via the server device 101 is displayed, along with a message prompting viewers of the lecture to begin a video conference discussion, and an image notifying them of the remaining time for the discussion. Furthermore, in area A2 of the streaming page illustrated in Figure 3(C), in addition to the "End" button, a "Go to Video Conference" button is displayed.

[0081] When a user of the receiving device 12 clicks the "Join Video Conference" button on the streaming page shown in Figure 3(C), the streaming page is closed according to the code associated with the "Join Video Conference" button (for example, a program written in Java® script, etc.), and request data is sent from the receiving device 12 to the server device 102 requesting permission to join the video conference. This request data includes a user ID that identifies the user of the receiving device 12.

[0082] When server device 102 receives request data from destination device 12, it sends request data to server device 103 requesting the group number of the group to which the user of destination device 12, the source of the request data, belongs. This request data includes the user ID that server device 102 received from destination device 12. Server device 103 searches the user information table for a data record corresponding to the user ID included in the request data received from server device 102, and sends the group number stored in the "Group Number" column of the searched data record to server device 102 as a response to the request data.

[0083] Server device 102 determines whether it has already started generating audiovisual data for video conferencing corresponding to the group number received from server device 103. If the generation of audiovisual data for video conferencing corresponding to that group number has not yet started, server device 102 starts generating the audiovisual data for video conferencing corresponding to that group number.

[0084] Next, the server device 102 sends request data to the destination device 12, the source of the request data, requesting it to start transmitting audiovisual data. Upon receiving the request data from the server device 102, the destination device 12 starts generating audiovisual data that represents a video with sound, including images captured by a camera connected to its device and sound picked up by a microphone connected to its device, and sequentially transmits the generated audiovisual data to the server device 102.

[0085] The server device 102 generates audiovisual data for video conferencing corresponding to the group number of the group in which the participants belong, using audiovisual data transmitted from each of the recipient devices 12 of the users belonging to that group, and audiovisual data for streaming distribution that is distributed from the source device 11 to the server device 102 via the server device 101.

[0086] The server device 102 sends display instruction data to the distribution destination device 12, the source of the request data, instructing it to display a web page for playing audiovisual data for video conferencing (hereinafter referred to as the video conference room page). The distribution destination device 12 displays the video conference room page shown in Figure 3(D) on its display device in accordance with the display instruction data received from the server device 102.

[0087] Area A3 of the video conferencing page displays images represented by audiovisual data for streaming. Area A4 of the video conferencing page displays images represented by audiovisual data transmitted from the distribution destination device 12 to the server device 102 for participants in this virtual conference room video conference.

[0088] While the video conferencing page is displayed, the distribution device 12 causes the sound-producing device (speaker) to emit the audio represented by the audiovisual data for video conferencing. The audio represented by the audiovisual data for video conferencing is the mixed audio of the voices of the video conference participants in this virtual conference room.

[0089] Users can discuss with other participants in the same group in a virtual meeting room by viewing images displayed on the video conferencing page and listening to audio spoken in conjunction with the display on the video conferencing page.

[0090] Area A5 of the video conferencing page displays virtual controls that accept user input. The controls displayed in Area A5 change, for example, in response to user input on the source device 11. In the video conferencing page illustrated in Figure 3(D), Area A5 includes a "Go to Streaming" button (an example of a control for ending participation in a video conference) to accept instructions to close the video conferencing page and open the streaming page, and a "Go to Small Group" button to accept instructions to open a video conferencing page for discussions among participants belonging to a small group, instead of the currently displayed video conferencing page.

[0091] When a user of the receiving device 12 clicks the "Go to Streaming" button on the video conference page shown in Figure 3(D), the video conference page is closed according to the code associated with the "Go to Streaming" button, and a request data is sent from the receiving device 12 to the server device 101 requesting the distribution of audiovisual data for streaming. In response to the request data received from the receiving device 12, the server device 101 sends display instruction data to the receiving device 12, instructing it to display the streaming page. The receiving device 12 then displays the streaming page on its display device, for example, as shown in Figure 3(B), according to the display instruction data received from the server device 101. In this way, the user of the receiving device 12 can leave the video conference they were participating in, return to the streaming page they were viewing before joining the video conference, and view the continuation of the lecture by speaker K (for example, Part 2).

[0092] When a user of the destination device 12 presses the "To Small Group" button on the video conference room page shown in Figure 3(D), the previously displayed video conference room page is closed according to the code associated with the "To Small Group" button, and request data is sent from the destination device 12 to the server device 102 requesting permission to join the small group video conference. This request data includes a user ID that identifies the user of the destination device 12.

[0093] When server device 102 receives request data from destination device 12, it sends request data to server device 103 requesting the group number of the subgroup to which the user of destination device 12, the source of the request data, belongs. This request data includes the user ID that server device 102 received from destination device 12. Server device 103 searches the user information table for a data record corresponding to the user ID included in the request data received from server device 102, and sends the group number stored in the "Subgroup Number" column of the retrieved data record to server device 102 as a response to the request data.

[0094] Server device 102 determines whether it has already started generating audiovisual data for video conferencing corresponding to the group number received from server device 103. If the generation of audiovisual data for video conferencing corresponding to that group number has not yet started, server device 102 starts generating audiovisual data for video conferencing for the subgroup corresponding to that group number.

[0095] Next, the server device 102 sends display instruction data to the distribution destination device 12, the source of the request data, instructing it to display a video conference room page for playing audiovisual data for small group video conferencing. The distribution destination device 12 displays the video conference room page shown in Figure 3(E) on its display device in accordance with the display instruction data received from the server device 102.

[0096] In the video conference page shown in Figure 3(E), area A4 displays images of participants belonging to small groups. While this video conference page is displayed, audio is played that is a mix of the voices of the participants whose images are displayed in area A4.

[0097] In the video conferencing page shown in Figure 3(E), area A5 includes a "Go to Streaming" button to accept instructions to close the video conferencing page and open the streaming page, and a "Go back to original group" button to accept instructions to open the video conferencing page of the group before it was divided into smaller groups, instead of the currently displayed video conferencing page.

[0098] The operation of the distribution destination device 12 and system 10 when the "Streaming Distribution" button on the video conferencing page shown in Figure 3(E) is pressed is the same as the operation of the distribution destination device 12 and system 10 when the "Streaming Distribution" button on the video conferencing page shown in Figure 3(D) is pressed.

[0099] Furthermore, when the "Return to Original Group" button on the video conferencing page shown in Figure 3(E) is pressed, the operation of the distribution destination device 12 and system 10 is the same as when the "Return to Subgroup" button on the video conferencing page shown in Figure 3(D) is pressed, except that the group number of the group before it was divided into subgroups is used instead of the group number of the subgroup.

[0100] In other words, when the "Streaming" button on the video conferencing page shown in Figure 3(E) is pressed, the streaming page shown in Figure 3(B) is displayed on the screen of the display device connected to the destination device 12. Also, when the "Return to Original Group" button on the video conferencing page shown in Figure 3(E) is pressed, the video conferencing page shown in Figure 3(D) is displayed on the screen of the display device connected to the destination device 12.

[0101] For example, during a lecture by speaker K, it may be necessary to gather feedback from users watching the lecture. In such cases, the user of the source device 11 performs a predetermined operation to instruct the source device 11 to proceed to answering a questionnaire. In response to this operation, the source device 11 sends predetermined instruction data to the server device 101. Upon receiving this instruction data, the server device 101 sends display instruction data to the destination device 12, instructing it to display a streaming page with a "Go to Questionnaire" button added to area A2. As a result, the screen of the display device connected to the destination device 12 displays the streaming page shown in Figure 3(F).

[0102] In the streaming page illustrated in Figure 3(F), area A1 displays a message encouraging viewers of the lecture to participate in a survey, along with an image notifying them of the remaining time to complete the survey. This content is being distributed from the source device 11 via the server device 101. Additionally, area A2 of the streaming page illustrated in Figure 3(F) displays "End" and "Go to Survey Form" buttons.

[0103] When a user of the receiving device 12 clicks the "Go to Questionnaire" button on the streaming page shown in Figure 3(F), the streaming page is closed according to the code associated with the "Go to Questionnaire" button, and request data is sent from the receiving device 12 to the server device 103 requesting the questionnaire page, which is a web page displaying the questionnaire.

[0104] When the server device 103 receives request data from the distribution destination device 12, it sends display instruction data, which is stored in advance, to the distribution destination device 12 that sent the request data, instructing it to display the survey page. The distribution destination device 12 displays the survey page shown in Figure 3(G) on the display device according to the display instruction data received from the server device 103.

[0105] Area A6 of the survey page displays multiple questions and input fields for answering those questions. For example, in addition to the input fields, the remaining time may also be displayed in Area A6. Furthermore, Area A7 of the survey page displays a "Submit" button and a "Go to Streaming" button.

[0106] When a user on the distribution device 12 enters their answers into the input fields on the survey page and then presses the "Submit" button, the distribution device 12 sends data indicating the user's answers, along with the user's user ID, to the server device 103. The server device 103 stores the data indicating the answers received from the distribution device 12 in the "Attributes" field of the data record in the user information table corresponding to the user ID received along with the data.

[0107] When a user of the distribution destination device 12 presses the "Go to Streaming Distribution" button on the survey page, the survey page is closed according to the code associated with the "Go to Streaming Distribution" button, and the streaming distribution page shown in Figure 3(B) is displayed on the display device screen. The operation of the system 10 and the distribution destination device 12 at that time is the same as the operation of the system 10 and the distribution destination device 12 when, for example, the "Go to Streaming Distribution" button on the video conferencing room page shown in Figure 3(D) is pressed.

[0108] After the participants have completed their survey as described above, the user of the source device 11 sends data to the server device 103, for example, using the source device 11, indicating new conditions for grouping based on the survey responses. The server device 103 groups the users of the destination device 12 into multiple groups so as to satisfy the conditions indicated by the data received from the source device 11, and updates the information in the "Group Number" and "Subgroup Number" columns of the user information table with information corresponding to the new groups.

[0109] Subsequently, when a user performs an operation such as pressing the "Go to Video Conference" button on a streaming distribution page, for example, as shown in Figure 3(C), the video conference page shown in Figure 3(D) will be displayed on the screen of the display device connected to the user's distribution destination device 12. However, the faces of the users participating in this video conference will be different from those of the users who participated in the previous video conference.

[0110] According to the system 10, source device 11, and destination device 12 described above, after a user watches the streamed content, they can discuss the content with other users in different locations (places where they cannot meet in person) who have also watched the same content. Furthermore, after finishing their discussion with other users, the user can watch the streamed content again.

[0111] [Differentiation] The embodiments described above can be modified in various ways within the scope of the technical idea of ​​the present invention. These modifications are shown below.

[0112] (1) In the embodiments described above, the destination device 12 does not display the streaming distribution page, the video conference room page, and the survey page simultaneously on the display device. However, the present invention is not limited to this point. Therefore, for example, the destination device 12 may display the video conference room page and the streaming distribution page simultaneously on the display device, or display the survey page and the streaming distribution page simultaneously.

[0113] If the destination device 12 displays the video conference page and the streaming distribution page simultaneously, the bitrate used when transmitting audiovisual data for video conferencing from the server device 102 to the destination device 12 for displaying the video conference page may be different from the bitrate used when transmitting audiovisual data for streaming distribution from the server device 101 to the destination device 12 for displaying the streaming distribution page.

[0114] Figure 4 shows the state where the streaming distribution page (left) and the video conferencing room page (right) are displayed side by side on the screen of a display device connected to the distribution destination device 12. In this case, the video conferencing room page does not necessarily have to display the images represented by the audiovisual data for streaming distribution that is streamed from the distribution source device 11.

[0115] For example, as shown in Figure 4, if the information displayed in area A8 of a streaming page consists almost entirely of text messages, then it is acceptable for the image displayed in area A8 to be blurry, as long as the user can easily read the text messages. On the other hand, the images of video conference participants displayed in area A9 of a video conferencing page should be as clear as possible so that participants can easily understand the nonverbal visual information (gestures, facial expressions, etc.) being transmitted by each other.

[0116] Therefore, for example, the streaming distribution means of server device 101 may be configured to distribute streaming audiovisual data to destination devices 12 that receive audiovisual data for video conferencing from the video conferencing relay means of server device 102 at a lower bitrate than to destination devices 12 that do not receive audiovisual data for video conferencing from the video conferencing relay means.

[0117] According to this modified version, for example, participating users can discuss the content of ongoing lectures, etc., on the streaming page with other participating users via video conference.

[0118] In this modified version, the host user may simultaneously switch the microphones of all participants in the video conference group on and off. For example, if the host user wants all participants to focus on watching the streamed content, they can turn off the microphones of all participants in the video conference. On the other hand, if the host user wants participants to freely discuss and debate the streamed content within their respective groups, they can turn on the microphones of all participants in the video conference.

[0119] Furthermore, in the above explanation, it was assumed that area A8 of the page shown in Figure 4 displays an image represented by the audio-video data transmitted from the server device 101 to the destination device 12, and area A9 displays an image represented by the audio-video data transmitted from the server device 102 to the destination device 12. However, instead, the image displayed in area A8 may be transmitted from the source device 11 to the destination device 12 via the server device 102. That is, similar to the video conference room page exemplified in Figure 3(D), the host user may participate in all conference rooms as a participant, and the audio-video transmitted from the source device 11 may be combined with the audio-video transmitted from the destination device 12 in the server device 102 to form the page format shown in Figure 4 for each conference room.

[0120] (2) In the embodiment described above, server devices 101, 102, and 103 transmit display instruction data instructing the display of a web page to the destination device 12, thereby causing the streaming page, video conference room page, or survey page to be displayed on the screen of the display device connected to the destination device 12. The destination device 12 then interprets the display instruction data received from server device 101, 102, or 103 using a web browser and displays the streaming page, video conference room page, or survey page on the display device in web page format.

[0121] The format of the data that the receiving device 12 receives from server device 101, server device 102, or server device 103 for displaying streaming pages, video conference rooms, or survey pages is not limited to display instruction data that instructs the display of a web page. Furthermore, the format of the streaming pages, video conference rooms, or survey pages that the receiving device 12 displays on its display device is not limited to the format of a web page.

[0122] For example, the receiving device 12 may, instead of using a web browser, follow a dedicated program to switch and display the streaming page, the video conferencing page, and the survey page according to the user's operation. Alternatively, the receiving device 12 may display one or two of the streaming page, the video conferencing page, and the survey page according to a dedicated program, and display the remaining pages using a web browser.

[0123] (3) In the embodiment described above, the system 10 is composed of server device 101, server device 102, and server device 103. The configuration of the system 10 is not limited to this. For example, one server device may perform two or more roles among server device 101, server device 102, and server device 103. For example, one computer may perform the role of a device that includes streaming distribution means for server device 101 and video conferencing relay means for server device 102 by executing a program.

[0124] (4) In the embodiment described above, the distribution destination device 12 displays the questionnaire page on its display device in response to an operation on the "Go to Questionnaire" button displayed on the streaming distribution page. Alternatively, the video conference page may include a "Go to Questionnaire" button, and the distribution destination device 12 may display the questionnaire page on its display device in response to an operation on the "Go to Questionnaire" button displayed on the video conference page. In this case, opinions and other information can be obtained from the user while the user is participating in the video conference.

[0125] (5) In the embodiment described above, while participating in the video conference, the user of the receiving device 12 (hereinafter referred to as the "participant user") can learn of messages from the user of the distributing device 11 (hereinafter referred to as the "organizer user") (for example, the topics to be discussed in the video conference or the remaining time for discussion) from the image represented by the audiovisual data for streaming displayed in area A3 of the video conference room page (see Figure 3(D)).

[0126] The method by which a host user can communicate messages to participants in a video conference is not limited to displaying images represented by streaming audiovisual data on the video conference page.

[0127] For example, information entered by the organizer user to the source device 11, or information such as the remaining time counted down by a timer on the source device 11, may be sent from the source device 11 to the server device 102 in the form of text data. The server device 102 may then include the contents of the text data received from the source device 11 in display instruction data that instructs the display of the video conference room page, thereby ensuring that messages from the organizer user to participant users are conveyed to the video conference room page displayed on the display device connected to the destination device 12.

[0128] Figure 5 shows an example of the screens displayed by the display devices connected to the source device 11 and the destination device 12 in this modified example. Figure 5(A) shows the screen displayed on the display device connected to the source device 11 for the host user to enter a message and remaining time for participant users participating in the video conference (hereinafter referred to as the "in-conference message input screen"). Figure 5(B) shows the video conference room page displayed on the display device connected to the destination device 12.

[0129] Area A10 of the meeting message input screen displays a text box for the host user to enter messages. Area A11 displays a text box for entering the remaining time and radio buttons for the host user to select whether to turn the display of the remaining time on or off.

[0130] Furthermore, the numerical value representing the time entered in the text box in area A11 will decrease (count down) automatically according to the clock on the source device 11 as time progresses, until it reaches 0 seconds remaining.

[0131] Additionally, Area A12 displays "Start" and "End" buttons, which accept commands to start and stop the display of messages and other information on the video conferencing page.

[0132] From the time the organizer user presses the "Start" button until they press the "End" button, each time the text data entered in area A10 is updated, that text data is sent from the distribution device 11 to the server device 102. In addition, if "ON" is selected in the radio button for area A11, from the time the organizer user presses the "Start" button until they press the "End" button, each time the text data indicating the remaining time, which is updated every second in area A11, is updated, the text data is sent from the distribution device 11 to the server device 102.

[0133] The server device 102 generates display instruction data for the video conference room page, which includes the text data transmitted from the source device 11 as described above, and sends it to the destination device 12. The destination device 12 displays the video conference room page shown in Figure 5(B) on its display device according to the display instruction data received from the server device 102. The video conference room page in this modified example includes an area A13 for displaying a text box, in which a message from the host user and the remaining time are displayed.

[0134] According to this modified version, during a video conference, the information that the host user wants to convey to the participant users is transmitted as text data rather than image data, thus avoiding unnecessary data transmission.

[0135] (6) In the embodiments described above, a participant user participating in a video conference closes the video conference page and reopens the streaming page by clicking the "Go to Streaming" button displayed on the video conference page. Therefore, even after the time for the video conference has ended and streaming of new content (for example, the continuation of the lecture) has started on the streaming page, if the participant user does not notice this and clicks the "Go to Streaming" button on the video conference page, the participant user will miss the streaming content.

[0136] To address such inconveniences, the video conferencing page may be configured to automatically switch from the video conferencing page to the streaming page once the allotted time for the video conference has ended, without waiting for any action from the participating users.

[0137] Figure 6 illustrates a video conference room page displayed on a display device connected to the destination device 12 in this modified example. In this modified example, for example, the video conference room page displays a countdown timer for the remaining time for the video conference in a text box in area A13, similar to the case in modified example (5). Then, according to the code associated with that text box, when the remaining time reaches a predetermined time (for example, 60 seconds), a window displaying a countdown timer for the remaining time pops up on the video conference room page, as shown in Figure 6.

[0138] When the remaining time displayed in this countdown reaches 0 seconds, the video conferencing page will switch to the streaming page display, just as if the "Go to Streaming" button had been operated, according to the code associated with the text box in area A13.

[0139] Furthermore, the system may be configured so that the volume of the video conference audio emitted from the speaker connected to the distribution device 12 gradually decreases from the time the remaining time reaches a predetermined amount until it reaches 0 seconds. Additionally, display switching effects may be used, such as fading out the video conference page while fading in the streaming distribution page from the time the remaining time reaches a predetermined amount until it reaches 0 seconds.

[0140] According to this modified version, participating users will not continue the video conference after the allotted time has ended, thus preventing them from missing the streamed content.

[0141] (7) The organizer user may be made able to check the status of the participant users. In this modified example, when the organizer user performs a predetermined operation on the source device 11, a screen as shown in Figure 7 (hereinafter referred to as the "participant status display screen") is displayed on the display device connected to the source device 11.

[0142] The source device 11 requests display instruction data from the server device 103 in response to a predetermined operation by the organizer user, instructing it to display the participant status display screen. In response to this request, the server device 103 aggregates the data stored in the "Status" column of the user information table (Figure 2), generates display instruction data for the participant status display screen using the aggregated results, and sends it to the source device 11. For example, each time the data in the "Status" column of the user information table is updated, the server device 103 generates display instruction data for the participant status display screen that reflects the updated data and sends it to the source device 11.

[0143] The source device 11 displays the participant status display screen on the display device in accordance with the display instruction data transmitted from the server device 103. The participant status display screen, as illustrated in Figure 7, displays the number of online users, the ratio (percentage) of the number of users receiving the streaming broadcast to the number of online users, and the ratio (percentage) of the number of users participating in the video conference to the number of online users.

[0144] The host user can view the participant status display screen and see in real time situations such as, for example, that many participants are still continuing the video conference even though the allotted time for the video conference has ended, and are not watching the streaming broadcast. Therefore, if many participants are still continuing the video conference, the host user can delay the start of streaming new content and prompt participants who are still continuing the video conference to end it by displaying a message on the video conference page or by making an audio call.

[0145] (8) The video conference page may be configured to display the status of participant users belonging to the group who have not yet joined the video conference. Figure 8 shows the video conference page displayed by the display device connected to the distribution destination device 12 in this modified example. Area A4 of the video conference page shown in Figure 8 is prepared for displaying the image of each participant user in the group, regardless of whether they are participating in the video conference or not. In these areas, the image of a participant user who has already joined the video conference is displayed, and the status of a participant user who has not yet joined the video conference is displayed.

[0146] For example, whenever the data in the "Status" column of the user information table (Figure 2) is updated, server device 103 sends the data in the "User ID," "Status," and "Group Number" columns of the user information table to server device 102. When server device 102 receives this data from server device 103, it generates display instruction data for the video conference room page that reflects this data and sends it to the distribution destination device 12. As a result, the status of participant users who have not yet joined the video conference is displayed in real time on the display device connected to the distribution destination device 12.

[0147] According to this modified version, participants already in a video conference can see how many other participants in the same group are not yet in the video conference, and whether those participants are currently offline or watching a streaming broadcast. Therefore, participants in a video conference can determine whether all available participants are present. Thus, participants in a video conference can decide whether to start the discussion or wait for other participants who are not yet in attendance but are expected to join soon.

[0148] In area A4 of the video conferencing page, a message from a participant user to other participants may be displayed in place of, or in addition to, the status of a participant user who has not yet joined the video conference. In this case, the user information table (Figure 2) may have a "Message" column, and any message entered by each participant user by operating the distribution destination device 12 (for example, "I will be late. I plan to join around 14:00.") is sent to the server device 103 and stored in the "Message" column of the user information table. Then, the server device 102 uses the data in the "Message" column received from the server device 103 to generate display instruction data to display the message in area A4 and sends it to the distribution destination device 12. As a result, participants who are currently in the video conference can receive messages from participants who are not currently in the video conference via the video conferencing page.

[0149] In this modified example, messages entered by participant users are sent to server device 102 via server device 103 and notified to participants in the video conference. Therefore, even if, for some reason, the destination device 12 is unable to access server device 102 and participate in the video conference, the message can still be conveyed to other participants in the video conference.

[0150] (9) In a video conference, participants may be configured to share their screens, exchange files, send and receive text messages (so-called chat).

[0151] (10) The system may be configured so that each participant user can check their own status history. Figure 9 is an example of a screen (hereinafter referred to as the "status history display screen") for participant users to check their own status history. When a participant user performs a predetermined operation on the distribution destination device 12, the distribution destination device 12 makes a request to the server device 103, and in response to that request, the server device 103 generates display instruction data for the status history display screen and sends it to the requesting distribution destination device 12. The distribution destination device 12 displays the status history display screen on the display device according to the display instruction data sent from the server device 103.

[0152] Area A14 of the status history display screen shows a bar graph indicating the time periods during which the organizer streamed content (excluding streaming content for displaying messages or remaining time during the video conference) and the time periods allocated for the video conference. Additionally, area A15 of the status history display screen shows a bar graph of the participant users' status history.

[0153] For example, server device 103 records the start and end times of streaming distribution from server device 101 to distribution destination device 12, as well as the standard start and end times of all video conferences in which the host user participates as a participant. Server device 103 also records the start and end times for displaying the streaming distribution page and the video conference room page for each participant user. Based on this recorded information, server device 103 generates display instruction data for the status history display screen and transmits it to distribution destination device 12.

[0154] In the bar displayed in Area A14, the time periods during which the organizer streamed content but participant users were not receiving the stream will be distinguished from the time periods during which the stream was received and displayed in a different manner (for example, a different color).

[0155] In the status history display screen, area A14 shows bars that indicate the time period of streaming delivery. These bars are associated with recorded data of content that was streamed during the time period indicated by that bar.

[0156] The streamed content is generated, for example, in server device 101, transmitted from server device 101 to server device 103, and stored in server device 103.

[0157] When a participant user performs a predetermined operation, such as clicking with the mouse, on the time period for which they want to play the recorded data, the recorded data for that time period (recorded data representing the streamed content) is sent from the server device 103 to the distribution destination device 12. At the same time, the image represented by the image data contained in the recorded data is displayed in area A16 of the status history display screen, and the sound represented by the sound data contained in the recorded data is played from the speaker connected to the distribution destination device 12.

[0158] Participant users can stop playback of recorded data by operating the "Stop" button displayed in area A17 of the status history display screen, or resume playback of stopped recorded data by operating the "Play" button.

[0159] According to this modified version, participating users can easily find out when they missed the streamed content. They can then watch the missed content.

[0160] (11) If new content is streamed while a participant user is continuing a video conference, the system may be configured to allow the recorded new content to be played back when the participant user returns to watch the streamed content.

[0161] Figure 10 illustrates the relationship between the time period during which the organizers stream content in real time and the time period during which participating users view that content in this modified example.

[0162] In the example shown in Figure 10, the organizer streams Content 1 from time t1 to time t2, allocates the time from time t2 to time t3 for video conferencing, streams Content 2 from time t3 to time t4, allocates the time from time t4 to time t5 for video conferencing, allocates the time from time t5 to time t6 for a break, and streams Content 3 from time t6 to time t7.

[0163] In contrast, a participant user in a certain group watched Content 1 in real time from time t1 to time t2, and then joined a video conference that started at time t2. However, the video conference was prolonged and continued until time t8, after time t3. In this case, when this participant user returns to watching the streaming broadcast, the server device 101 streams recorded data of Content 2 to the destination device 12, replacing the data of Content 2 that is being transmitted in real time from the source device 11 to the server device 101. Therefore, the time when the participant user finishes watching Content 2 is time t9, after time t4.

[0164] A participant user joins the video conference at time t9. In this case, other participant users in the same group will also view content 2 from time t8 to time t9 and then join the video conference at time t9.

[0165] Subsequently, the participants in this group finish their video conference at time t10, later than the other groups. After taking a short break, they begin viewing content 3 at time t6. In this case, the data for content 3, which is being transmitted in real time from the source device 11 to the server device 101, is streamed from the server device 101 to the destination device 12.

[0166] According to this modified version, even if the end times of video conferences differ for each group, participating users will not miss out on watching the streamed content.

[0167] (12) In the embodiments described above, the members of the small group in the video conference are pre-configured and registered in the user information table (Figure 2). Alternatively, or in addition to this, a configuration may be adopted in which a participant user can decide which members of a small group, including themselves, are to be included in at any time during the video conference, and start a video conference for that small group with the decided members.

[0168] Figure 11 illustrates the video conference room page displayed on the display device connected to the distribution destination device 12 in this modified example. Figure 11(A) shows the video conference room page in a state where all participant users with the same group number stored in the "Group Number" column of the user information table (Figure 2) are participating in the same video conference. In the state shown in Figure 11(A), for example, if participant user A wants to have a separate video conference with participant user E, participant user A will operate on the "Select Member" button displayed in area A5 of the video conference room page, and then operate on the image of participant user E displayed in area A4 to select participant user E. Note that the number of members with whom you want to have a separate video conference is not limited to one; after operating on the "Select Member" button, you may select two or more members.

[0169] After participant user A selects the person they wish to have a separate video conference with as described above, they perform an action on the "To Small Group" button displayed in area A5. In response to this action, the display devices connected to the destination device 12 for participant user A and participant user E will display the video conference room page as shown in Figure 11(B). Additionally, the display devices connected to the destination device 12 of the other participant users will display the video conference room page as shown in Figure 11(C).

[0170] In area A5 of the video conferencing page shown in Figure 11(C), a "Help" button is displayed. This "Help" button is used to request the host to add new participants if some members of the group leave the video conference. For example, if participant A is the moderator of this group's video conference, and for some reason participant A feels the need to have a separate discussion with participant E and starts a small group video conference with participant E, the remaining participants will be without a moderator and will find it difficult to continue the video conference. In such a case, one of the remaining participants (for example, participant B) can use the "Help" button to request the participation of a participant who can take the place of participant A.

[0171] In response to the operation of the "Help" button, for example, request data is sent from the distribution destination device 12 to the server device 103 requesting the participation of a new participant user. This request data includes the group number of the group to which the user of the sending distribution destination device 12 belongs. In this modified example, the user information table (Figure 2) stored in the server device 103 has a "Role" column, for example, and data such as "Moderator" is stored in the "Role" column for participant users who can serve as moderators. Some of the participant users who are moderators are not assigned a group number, and the data "Waiting online" is stored in the "Status" column. These participant users wait without participating in any video conferences while other participant users are participating in video conferences.

[0172] When the server device 103 receives request data from the destination device 12, it sends a message requesting participation in the video conference (for example, "Group XX has requested that the moderator join the conference.") to each of the destination devices 12 of the participant users whose "Status" column in the user information table is "Waiting online". The display device connected to the destination device 12 of the waiting participant user displays the message sent from the server device 103 and a "Join Video Conference" button.

[0173] The first participant user among those waiting to click the "Join Video Conference" button will join the video conference of the requesting group. Once any participant user joins the video conference in response to the request, the display devices connected to the distribution destination device 12 of the other waiting participant users will display a message such as, "Another member has joined the requested video conference."

[0174] According to this modified version, individual video conferences can be held depending on the progress of the video conference.

[0175] (13) In the embodiment described above, all participant users registered in the user information table become members of one of the groups and participate in the video conference. Alternatively, participant users who are absent from the video conference may be configured not to become members of any group.

[0176] In this modified example, before the video conference is held, each participant user can access the server device 103 using, for example, the distribution destination device 12, and register with the server device 103 whether or not they will participate in the video conference. One of the "Attribute 1" to "Attribute m" columns in the user information table stored in the server device 103 is assigned to a field that stores data indicating whether or not they will participate in the video conference. Hereinafter, we will assume that "Attribute 10" is that field as an example.

[0177] The server device 103 stores data indicating whether or not a participant can join the video conference transmitted from the distribution destination device 12 in the "Attribute 10" field. When the server device 103 groups video conferences, participant users whose "Attribute 10" field contains data indicating they cannot join the video conference are excluded from the grouping and are not assigned a group number.

[0178] Alternatively, instead of using pre-registered data indicating whether or not a participant can join a video conference, the grouping process may include only participant users whose "Status" column in the user information table contains data indicating that they are receiving a streaming broadcast.

[0179] According to this modification, video conferences do not result in situations where many participants are absent, unintentionally forcing small groups to participate in discussions.

[0180] Furthermore, if a participant user whose group number has not been assigned comes online during a video conference, that participant user may be assigned a group number for one of the video conferences and allowed to join that video conference.

[0181] In that case, the group number to be assigned to participant users who have not yet been assigned a group number will be determined based on the number of people in each already grouped group, the attributes of the members belonging to each group, etc.

[0182] (14) When the server device 103 groups the video conference participants, it may assign a role such as moderator to one of the members belonging to each group.

[0183] In this modified example, the user information table (Figure 2) includes, for example, a "Role" column. After grouping, the server device 103 selects, for each group, participant users from among those belonging to that group, either randomly or based on data stored in the "Attributes" column of the user information table that meet predetermined conditions, and assigns a predetermined role (for example, facilitator) to the selected participant users. The server device then stores data indicating the assigned role in the "Role" column of the record of the participant user to whom the role was assigned in the user information table.

[0184] Figure 12 illustrates the video conference room page displayed on the display device connected to the destination device 12 in this modified example. As shown in Figure 12, in this modified example, the roles of the participant users are displayed in area A4 of the video conference room page.

[0185] According to this variation, the organizer does not need to assign roles to participant users in the video conference. Furthermore, participants can easily see their own and other participants' roles in the video conference.

[0186] (15) The status of participant users' past participation in video conferences may be recorded, and when grouping new video conferences, the grouping may be done based on the recorded past participation status of video conferences.

[0187] In this modified example, for instance, a "Participation History" column is provided in the user information table (Figure 2) stored in the server device 103. The "Participation History" column stores data indicating the time slots of past video conferences and the participant status of those video conferences.

[0188] When the server device 103 groups participant users for a new video conference, it determines the groups in such a way that they satisfy predetermined conditions regarding the data stored in the "Participation History" column.

[0189] The specified conditions for the data stored in the "Participation History" column include, for example, if the participation rate of all participating users in past video conferences is X%, then the condition is "the average participation rate of participating users in past video conferences for each group is (X-α)% or more and (X+α)% or less."

[0190] According to this variation, even if it is not possible to know in advance whether or not participants will be able to attend the video conference, the likelihood of a group having to unintentionally hold a discussion with a small number of people due to a large number of absentees is reduced.

[0191] (16) In a video conference held according to the groupings made before the video conference was held, if there are many absentees and the number of participants is small, participants may be moved from other groups with many attendees.

[0192] In this modified example, the server device 103, for example, after a predetermined time (e.g., 5 minutes) has elapsed from the start time of the video conference, refers to the user information table (Figure 2) and counts the number of participant users participating in the video conference for each group. Then, the server device 103 selects, for example, one participant user randomly from the participant users belonging to the group with the most participants, or selects a participant user whose attributes meet predetermined conditions, and changes the group number that was assigned to the selected participant user to the group number of the group with the fewest participants.

[0193] The server device 103 sends a message to the destination device 12 of the participant user whose group number has been changed, for example, "Your video conference group has changed. Please move to the video conference of the new group," along with a screen display instruction data that includes a "Move" button. In accordance with this display instruction data, a screen containing the above message and a "Move" button is displayed as a pop-up on the display device connected to the destination device 12 of the participant user whose group number has been changed. When the participant user clicks the "Move" button on that screen, the destination device 12 of that participant user will join the video conference identified by the new group number.

[0194] According to this modified version, if it is not possible to know in advance whether or not participants will be able to attend a video conference, and a group ends up having to hold a discussion with an unintended small number of people due to many absentees, a correction will be made.

[0195] (17) The display of both or one of the streaming page and the video conferencing page may be changed depending on the attributes of the participant user (e.g., language used, age, etc.).

[0196] In this modified example, server device 101 has pre-stored templates of display instruction data that instruct the server to display streaming pages in different display modes according to the attributes of the participating users. Server device 102 also has pre-stored templates of display instruction data that instruct the server to display video conferencing pages in different display modes according to the attributes of the participating users.

[0197] These templates include templates for different languages, such as "Japanese," "English," and "Portuguese," as well as templates for different age groups, such as "Children's" and "Adults'," and combinations thereof.

[0198] When the server device 101 sends display instruction data for a streaming page to the destination device 12, it receives data from the server device 103 indicating the attributes of the participant users using the destination device 12 (data stored in the "Attributes" column of the user information table), and generates display instruction data for a streaming page to send to the destination device 12 using a template corresponding to the attributes indicated by that data.

[0199] Furthermore, when the server device 102 transmits display instruction data for the video conference room page to the distribution destination device 12, it receives data from the server device 103 indicating the attributes of the participant users using the distribution destination device 12 (data stored in the "Attributes" column of the user information table), and generates display instruction data for the video conference room page to be transmitted to the distribution destination device 12 using a template corresponding to the attributes indicated by that data.

[0200] According to this modified version, for example, a participant user can view the streamed content on a streaming page that uses their preferred language, or participate in a video conference on a video conferencing page that uses their preferred language. Furthermore, according to this modified version, for example, if the participant user is young, they can view the streamed content on a streaming page that uses a lot of kana characters, or participate in a video conference on a video conferencing page that uses a lot of kana characters. Therefore, the participant user will not have trouble understanding the meaning of the various buttons displayed in area A2 of the streaming page or area A5 of the video conferencing page.

[0201] In this modified example, the server device 101 may, for example, convert the text portion of the image streamed from the source device 11 into text data using OCR, then machine translate it into the language used by the participant user, and stream the image with the machine-translated text to the destination device 12.

[0202] In this modified example, the server device 101 may convert the audio streamed from the source device 11 into text data using a known speech recognition method, then machine translate it into the language used by the participant user, synthesize an audio recording of the machine-translated text, and stream the synthesized audio to the destination device 12.

[0203] (18) As described in the modified example (14) above, the video conferencing page may be configured to display the attributes of a participant user on or near their image (see Figure 12). In the example in Figure 12, only for participant users whose "Role" column in the user information table (Figure 2) contains data indicating a specific role such as "Facilitator" or "Secretary," the text indicating the participant user's role is displayed on the participant user's image on the video conferencing page. Thus, instead of displaying attributes only for specific participant users, the system may be configured to display the attributes of all participant users.

[0204] Figure 13 illustrates a video conference page displayed on a display device connected to the destination device 12 in this modified example. As shown in Figure 13, in this modified example, the attributes of each participant user are displayed in area A4 of the video conference page. In the example in Figure 13, the displayed attributes are name, age, and occupation, but these are examples, and any other attributes such as self-introduction and place of residence may be displayed. These attributes may be registered in the user information table by the organizer user, for example, or information entered by participant users on the questionnaire page (see Figure 3(G)) may be registered in the user information table. Furthermore, the timing of the registration of these attributes in the user information table may be before the start of the video conference or after the start of the video conference.

[0205] (19) The images of participant users displayed on the video conference page, along with videos with audio representing the voices of those participant users, may be configured to be streamed to the distribution destination device 12 used by users who use the system 10 as observers (hereinafter referred to as observer users). For example, the organizer user may be an observer user, or the observer user may be registered as one of the attributes in the user information table (Figure 2), or all participant users may be allowed to view the streamed video conference as observer users.

[0206] In this modified example, either server device 101 or server device 102 may stream the video with audio from the video conference. When server device 101 streams the video with audio from the video conference, the video with audio transmitted from server device 102 to server device 101 is streamed from server device 101 to the observer user's destination device 12.

[0207] Figure 14 illustrates a page displayed on a display device connected to the observer user's distribution destination device 12 (hereinafter referred to as the observer page) in this modified example. Area A18 of the observer page displays a list of conference rooms that the observer user can view. Area A19 of the observer page displays images of participant users participating in the video conference in a conference room selected by the observer user from the conference rooms displayed in area A18. In addition, the speaker connected to the observer user's distribution destination device 12 plays the voice of the participant users participating in the video conference in the conference room selected by the observer user.

[0208] Figure 15 illustrates an example of a video conferencing page for a conference room being viewed by an observer user. The video conferencing page illustrated in Figure 15 displays the number of observer users currently viewing the conference room. This allows participants in a video conference to know if their conference room is being viewed by users who are not participating in the conference. Note that the indication that an observer user is viewing the conference room on the video conferencing page is not limited to text; it may also include graphics or other visual aids.

[0209] Furthermore, it is not necessary for the video conferencing page of a conference room being viewed by any observer user to display an indication that it is being viewed by an observer user. For example, if the host user monitors all video conferencing rooms so that they can quickly respond to any problems that occur in any of the video conferencing rooms, the video conferencing page does not necessarily need to display an indication that the host user is viewing that conference room.

[0210] Furthermore, the conference rooms that observer users can view may be all conference rooms that are open at that time, or they may be specific conference rooms selected, for example, by the host user.

[0211] (20) Participant users may want to consult with someone based on, for example, the content of a lecture delivered on the streaming page or a video conference held on the video conference page. For this purpose, system 10 may be configured so that a video conference is held between a participant user who wishes to consult (hereinafter referred to as the "consultation-seeking user") and a user who will receive the consultation (hereinafter referred to as the "consultation-providing user").

[0212] Figure 16 illustrates a streaming page displayed on a display device connected to the participant user's destination device 12 in this modified example. When a participant user operates the destination device 12 and clicks the "Consult" button displayed on the streaming page in Figure 16, the page shown in Figure 17 (hereinafter referred to as the consultation page) is displayed. The display instruction data for the consultation page is transmitted, for example, from the server device 102. The consultation page is a video conference room page in which the consultation requester and the consultation user participate. Area A20 of the consultation page displays images of the consultation user and the consultation requester. In addition, the consultation user's voice is played from the speaker connected to the consultation requester's destination device 12.

[0213] Figure 18 is an example of a page (hereinafter referred to as the "consultant page") displayed on a display device connected to the consultant user's distribution destination device 12. Area A21 of the consultant page displays the image of the user requesting consultation. Area A22 of the consultant page displays the image of the consultant user (the user). In addition, the voice of the user requesting consultation is played from the speaker connected to the consultant user's distribution destination device 12.

[0214] Area A22 of the advisor's page displays the advisor user's (the user's) image and their status. This status can be selected by the advisor user from options such as "On standby," "Away from desk," and "In progress." The status may be automatically set to "In progress" upon the start of a video conference between the advisor user and the user seeking advice. Similarly, the status may be automatically set to "On standby" upon the end of the video conference. The advisor user's status is stored as one of the attributes in the user information table (Figure 2).

[0215] Area A23 of the advisor page displays a list of other advisor users who are currently on standby. When an advisor user receives a consultation request from a user on the advisor page and determines that another advisor user is more suitable, they select the most suitable advisor user from the list of standby advisor users displayed in Area A23 of the advisor page. Figure 19 is an example of the advisor page when an advisor user is selected from the standby advisor users. Hereafter, the advisor user viewing the advisor page in Figure 18 (the advisor user whose image is displayed in Area A22) will be referred to as the referring advisor user, and the other advisor user selected by the referring advisor user will be referred to as the receiving advisor user.

[0216] Area A24 of the advisor's page displays an image of the advisor user to whom the call is being forwarded. In this state, the speaker connected to the distribution device 12 of the forwarding advisor user will play the voice of the forwarding advisor user instead of the voice of the user requesting the consultation.

[0217] While the advisor page shown in Figure 19 is displayed, a temporary video conference takes place between the transferring advisor user and the receiving advisor user. While the temporary video conference is taking place between the transferring advisor user and the receiving advisor user, the video conference between the transferring advisor user and the user requesting consultation is maintained. However, in the consultation page displayed to the user requesting consultation (Figure 17), area A20 may display a message such as "Please wait a moment" instead of the image of the transferring advisor user. In this case, it is desirable that the voice of the transferring advisor user is not emitted (muted) from the speaker connected to the receiving device 12 of the user requesting consultation.

[0218] Figure 20 illustrates the advisor page displayed on a display device connected to the distribution destination device 12 for a waiting advisor user. When an advisor user is waiting, no video conference is taking place with other users, so images of other users are not displayed in areas A21 and A24. However, the advisor user's own image is displayed in area A22.

[0219] Figure 21 illustrates the consultant page displayed when a waiting consultant user is selected as the destination consultant user by the transferring consultant user. In this state, the image of the transferring consultant user is displayed in area A24 of the consultant page. In addition, the voice of the transferring consultant user whose image is displayed in area A24 is played from the speaker connected to the distribution device 12 of the destination consultant user. In other words, a temporary video conference is taking place between the transferring consultant user and the destination consultant user. In this state, the consultant page shown in Figure 19 is displayed to the transferring consultant user.

[0220] The forwarding consultant user relays the information gathered from the participant user requesting consultation to the receiving consultant user. Based on this information, if the receiving consultant user decides to take over the consultation from the forwarding consultant user, the forwarding consultant user clicks the "End" button displayed between areas A21 and A22 on the consultant page (Figure 19). Upon this action, the video conference between the forwarding consultant user and the user requesting consultation ends, and a video conference between the receiving consultant user and the user requesting consultation begins.

[0221] In this case, the image of the receiving consultant will be displayed in area A20 of the consultation page (Figure 17) shown to the user requesting consultation, instead of the image of the referring consultant. Also, the voice of the receiving consultant will be played from the speaker connected to the receiving device 12 of the user requesting consultation, instead of the voice of the referring consultant.

[0222] Furthermore, the advisor page displayed to the receiving advisor user will be in the state shown in Figure 18. Specifically, an image of the user requesting consultation will be displayed in area A21. Then, the voice of the user requesting consultation will be played from the speaker connected to the receiving advisor user's distribution device 12.

[0223] In this example, the video conference between the sending consultant user and the user requesting consultation ends and the video conference between the receiving consultant user and the user requesting consultation begins, triggered by an action by the sending consultant user. Alternatively, the video conference between the sending consultant user and the user requesting consultation may end and begin when an action is taken by the receiving consultant user.

[0224] If the receiving consultant is unable to take over the consultation from the forwarding consultant for any reason, either the forwarding or receiving consultant clicks the "End" button displayed between areas A22 and A24 on the consultant page (Figures 19 and 21). This action terminates the video conference between the two forwarding consultants. At that time, if a message such as "Please wait a moment" was displayed in area A20 of the consultation page (Figure 17) that was shown to the user requesting consultation, the image of the forwarding consultant will be displayed instead of the image of the forwarding consultant. Also, if the forwarding consultant's audio was muted, it will be unmuted, and the forwarding consultant's voice will be played from the speaker connected to the consultation requester's destination device 12.

[0225] According to this modified version, a user seeking advice can consult with a user acting as an advisor. Furthermore, if there is another user more suitable as an advisor, the advisor can transfer the consultation to that user.

[0226] When a user requesting consultation clicks the "Consult" button on the streaming page shown in Figure 16, the system 10 (for example, server device 101, server device 102, or server device 103) needs to select a consultant user from among several waiting consultant users to respond to the consultation request (i.e., match the consultation request user with a consultant user). This selection method may be random, or a method may be used to select a consultant user such that the attributes of the consultation request user and the consultant user satisfy predetermined conditions.

[0227] Furthermore, when a user requesting consultation clicks the "Consult" button on the streaming page shown in Figure 16, a questionnaire page (see Figure 3(G)) may be displayed, and a consultant user may be selected if the answers (attributes) entered by the user requesting consultation on the questionnaire page and the attributes of the consultant user meet predetermined conditions.

[0228] Alternatively, the matching of users seeking consultation with consultant users may be performed by, for example, the organizer user, instead of system 10. In that case, for example, the organizer user's distribution device 11 will display a page like the one shown in Figure 22 (hereinafter referred to as the matching page). Area A25 of the matching page will display a list of users seeking consultation who have not yet been matched with a consultant user, along with their attributes and waiting time. The list displayed in area A25 will show users seeking consultation in order of longest waiting time.

[0229] Area A26 of the matching page lists the names of all currently online advisor users, along with their attributes, images, and status. The images of the advisor users displayed in Area A26 are images of the current appearance of each advisor user, captured by cameras connected to the distribution device 12 they are using. In addition, for advisor users whose status displayed in Area A26 is "Currently Handling," the names of the users requesting consultation who are currently in a video conference with that advisor user are also displayed in Area A26.

[0230] The host user selects matching users based on the attributes of the user seeking advice and the advisor user, and clicks the "Match" button. In Area 26, advisor users whose status is "Away" or "Currently Assisting" will have their selection boxes hidden (or displayed in a way that indicates they cannot be selected), and the host user will not be able to select them. Upon the host user clicking the "Match" button, a video conference will begin between the user seeking advice and the advisor user selected by the host user. Additionally, on the matching page, the matched user seeking advice will no longer be displayed in Area A25, and in Area A26, the status of the matched advisor user will change to "Currently Assisting," and the name of the user seeking advice that was no longer displayed in Area A25 will now be displayed as the name of the user seeking advice that the advisor user is currently assisting.

[0231] Furthermore, on the matching page, System 10 may recommend and display combinations of users who wish to consult and users who are willing to consult, based on their attributes. In this case, the organizer user can select the users who wish to consult and users who are willing to consult, referring to the combinations of users who wish to consult and users who are willing to consult displayed by System 10.

[0232] After a user requesting consultation clicks the "Consult" button on the page shown in Figure 16, that user will enter a consultation meeting room. Multiple users who have not yet been matched with a consultant can enter the consultation meeting room simultaneously. A host user is always present in the consultation meeting room. The cameras and microphones of users requesting consultation in the consultation meeting room are automatically turned off. The host user present in the consultation meeting room will then broadcast a message, such as "Please wait a moment," or a guidance video, to all users requesting consultation in the meeting room.

[0233] Figure 23 shows the consultation page (see Figure 17) displayed on the display device connected to the distribution destination device 12 for a user seeking consultation, before they are matched with an advisor user. Until an advisor user is matched, area A20 of the consultation page displays a message such as "Please wait a moment" or a guidance video from the organizer user, instead of the advisor user's image.

[0234] In this modified example, video conferences are held between multiple pairs of users seeking consultation and consulting users. The method by which system 10 generates conference rooms when these video conferences take place could include, for example, sequentially generating new conference rooms as needed, or pre-generating conference rooms for each consulting user and having the users seeking consultation enter the consulting user's conference room.

[0235] Additionally, a small meeting room in the consultation reception area may be used as a meeting room for video conferencing between users seeking consultation and the consulting users.

[0236] Furthermore, multiple meeting rooms for receiving consultations may be provided. In this case, the method for assigning new users who wish to receive consultations to one of the meeting rooms may be random, or by assigning them to a meeting room with a small number of waiting users. Also, if multiple meeting rooms with different attributes are provided, a configuration may be adopted in which new users who wish to receive consultations are assigned to a meeting room whose attributes match the attributes of the new user and meet the specified conditions.

[0237] (21) The page displayed on the display device connected to the source device 11 or the destination device 12 may be configured to display a message so that the user can easily confirm the content of the operation performed by the user. Figure 24 shows an example of a video conference room page displayed on the display device connected to the destination device 12 in this modified example. Area A5 of this video conference room page displays a "microphone" button and a "camera" button. By clicking the "microphone" button, the user can stop the microphone operation so that their voice cannot be heard by other video conference participants, or restart the microphone operation that was stopped so that their voice can be heard by other video conference participants. In addition, by clicking the "camera" button, the user can stop the camera operation so that their image cannot be seen by other video conference participants, or restart the camera operation that was stopped so that their image can be seen by other video conference participants.

[0238] The "microphone" and "camera" buttons are displayed in different ways depending on whether they are active or not, for example, green when active and red when inactive. However, users may not know which display means that the function is active. Also, even when different characters are displayed to distinguish between active and inactive states, such as "microphone ON" and "microphone OFF," it may be difficult for users to determine whether, for example, "microphone ON" indicates (1) that the microphone is currently ON (active), or (2) that it is a button to switch a currently OFF (inactive) microphone to ON (active).

[0239] To resolve the above issue, in this modified version, when the user performs an action on a button, a message explaining the meaning of that action is displayed. Figure 25 is an example of a video conferencing page where a message explaining the meaning of the click is displayed when the user clicks the "microphone" button. In the example in Figure 25, a message pops up indicating that the microphone has changed from inactive (OFF) to active (ON) after the user clicked the "microphone" button.

[0240] According to this modification, the user will not misunderstand the meaning of the actions they have performed.

[0241] (22) In the embodiment described above, the video conference page displays the streaming page distributed from the source device 11 via the server device 101 (see area A3 in Figure 3(D)). This means that in system 10, the host user, who is a user of the source device 11, is simultaneously participating in each of the multiple video conferences being held concurrently as one of the participants.

[0242] The information that a host user participating in multiple video conferences simultaneously wants to deliver to the participants of those conferences changes depending on the situation. Therefore, the system 10 may be configured so that the host user can appropriately select from multiple options the content to be simultaneously delivered from the source device 11 to multiple conference rooms.

[0243] Figure 26 is an example of a page displayed by a display device connected to the source device 11 in this modified example (hereinafter referred to as the video conferencing content selection page). The video conferencing content selection page displays multiple candidate content to be simultaneously distributed to multiple conference rooms. In the example in Figure 26, content C1, which distributes the image and audio of the host user, and content C2, which distributes a specific text message, are displayed as options. When the host user selects any of these options, the selected content is distributed from the source device 11 to the destination devices 12 of the participant users participating in the video conference, for example, via server devices 101 and 102.

[0244] (23) When multiple video conferences are held simultaneously by each of several groups, some participants may be absent, late, or leave early, resulting in some conference rooms having insufficient participants or lacking participants in essential roles (such as facilitators). In such cases, to eliminate conference rooms with insufficient participants, the host user may, for example, enable the consolidation of multiple conference rooms.

[0245] Figure 27 is an example of the page displayed by the display device connected to the source device 11 in this modified example (hereinafter referred to as the conference room integration page). The conference room integration page lists the group number of conference rooms with insufficient participants, the number of participants, and whether or not there are participants with a specific attribute (in the example of Figure 27, the facilitator). The number of participants refers to the number of people who are scheduled to participate before the start of the video conference, and the number of people who are actually participating after the start of the video conference. However, participants who are scheduled to participate but will be late, or participants who have temporarily left but will return, may also be included in the number of participants.

[0246] The meeting rooms listed on the meeting room integration page are those that have been extracted by system 10 (for example, server device 103) as meeting rooms that satisfy predetermined conditions (for example, the number of participants is 3 or less, the participant acting as facilitator is absent, etc.) where at least one of the number of participants or the attributes of the participants is met.

[0247] The organizer user selects multiple meeting rooms whose participant shortages will be resolved by merging, based on the information listed on the meeting room merging page, and clicks the "Merge" button. In response to this operation, the server device 103 updates the information in the "Group Number" column of the user information table (Figure 2), for example, to show the merged group number. Following this update, the server device 102 starts generating and transmitting a video conference room page containing audio and video of the participant users so that the participant users who were participating in the multiple meeting rooms before the merger will join the single merged meeting room.

[0248] Please note that information about the merged meeting rooms will no longer be displayed in the list on the meeting room integration page.

[0249] In the example above, the organizer user selects the meeting rooms to be merged. Alternatively, system 10 may identify multiple meeting rooms that satisfy a predetermined condition (for example, four or more participants and one participant acting as a facilitator) in either the number of participants or the attributes of the participants after the merged meeting rooms. In this case, the multiple meeting rooms identified by system 10 are displayed as recommendations on the meeting room merging page, and the organizer user may select the meeting rooms to be merged based on these recommendations, or system 10 may merge meeting rooms it has identified independently of the organizer user's judgment.

[0250] The consolidation of meeting rooms may be carried out either before or after the start of the video conference.

[0251] (24) In some cases, some of the participating users may gather in one conference room and view a streaming page displayed on a large screen. In this case, the participating users gathered in the conference room may be allowed to participate in the video conference using the destination device 12. In this modified example, for example, "Onsite" is registered in the "Status" column of the user information table (Figure 2) to indicate that the participant user is viewing the streaming page in the conference room. This registration is performed, for example, by a user who is registering participants in the conference room operating the source device 11 or the destination device 12. Then, for example, when the server device 103 groups the participating users to meet predetermined conditions, the server device 103 includes users with "Onsite" registered in the "Status" column of the user information table in the grouping. This allows even participant users who do not view the streaming page using the destination device 12 to participate in the video conference.

[0252] (25) Participant users may be allowed to view information about other participant users who are having a video conference in the same conference room as them. Figure 28 is an example of a page displayed by a display device connected to the distribution destination device 12 (hereinafter referred to as the video conference participant display page) in this modified example. The video conference participant display page lists the names and attributes (age and occupation in the example of Figure 28) of users who are assigned the same group number as the user of the distribution destination device 12 that is displaying the page. For example, a user can obtain information about the person they will be having a video conference with by looking at the video conference participant display page before the start of the video conference.

[0253] (26) When the server device 103 groups participant users, it may group only participant users whose attributes registered in the user information table (Figure 2) meet predetermined conditions. For example, if the content to be discussed in the video conference is related to the content of a questionnaire that has been answered in advance by the participant users, the server device 103 may extract participant users whose answers to that questionnaire are registered as attributes in the user information table and group those extracted participant users.

[0254] (27) During a video conference, it may be possible to send and receive messages (so-called chat) between the host user and the participant user. In this case, a list of pre-written messages may be displayed to the host user or the participant user, and the selected pre-written message may be sent to the other party.

[0255] Additionally, the host user may enable sending messages simultaneously to all participating users in the video conference, or to all participating users with specific attributes (for example, participants whose role is "caretaker").

[0256] Furthermore, during a video conference, only participant users with specific attributes (for example, participants with the role of "caretaker") may be allowed to send messages to the host user. In this case, the message input field for the host user will only be displayed on the video conference page shown by the display device connected to the destination device 12 for participant users with specific attributes. Thus, the content displayed on the video conference page may be configured to differ depending on the attributes of the participant user.

[0257] In this modified example, for instance, the server device 102 relays data indicating messages being sent and received between the source device 11 and the destination device 12.

[0258] For example, during a video conference, a participant acting as a moderator may want to ask questions or make requests to the host user. Also, during a video conference, the host user may have information they want to convey to the moderator participants in each video conference. This modified version satisfies these needs.

[0259] (28) In the embodiments or modifications described above, operation buttons that are displayed on one page may be displayed on other pages. For example, the "Go to Questionnaire" button that is displayed on the streaming page in Figure 3(F) may be displayed on the video conferencing page. In this case, participant users can open the questionnaire page (Figure 3(G)) from the video conferencing page.

[0260] Furthermore, in the embodiments or modifications described above, a page listing the operation buttons that are to be displayed on pages with specific roles, such as streaming pages and video conferencing rooms (hereinafter referred to as the operation menu page), may be displayed to the user. The user can open the page they want to open by clicking an operation button on the operation menu page.

[0261] (29) The server device that transmits the display instruction data for a page may be any of server device 101, server device 102, and server device 103. Furthermore, system 10 may include a server device different from these server devices, and the display instruction data may be transmitted from that server device. Also, data for displaying a single page may be transmitted from multiple server devices to the source device 11 or the destination device 12. For example, if content streamed from source device 11 is displayed on a video conference room page, the destination device 12 may receive the content directly from server device 101 without going through server device 102. In this case, the destination device 12 may, for example, receive the content directly from server device 101 according to the display instruction data for the video conference room page received from server device 103, embed the received content on the video conference room page, and display it on the connected display device.

[0262] (30) In the embodiments or modifications described above, the tasks that the organizer user would normally perform using the source device 11 may be performed by the participant user using the destination device 12. For example, the task of integrating meeting rooms, which the organizer user would normally perform on the meeting room integration page in order to resolve the issue of insufficient participants in a meeting room, may be performed by a participant user who has been given the role of administrator or the like. [Explanation of Symbols]

[0263] 9...Communication network, 10...System, 11...Source device, 12...Destination device, 101...Server device, 102...Server device, 103...Server device.

Claims

1. A streaming distribution means that receives audiovisual data from a source device and streams the audiovisual data to multiple destination devices, A video conferencing relay means that relays a video conference by receiving audiovisual data from each of two or more destination devices participating in a video conference, which are among a plurality of destination devices receiving audiovisual data from the aforementioned streaming distribution means, receiving audiovisual data representing images captured by cameras connected to the destination devices and sound picked up by microphones connected to the destination devices, synthesizing the received audiovisual data to generate audiovisual data for video conferencing, and transmitting the generated audiovisual data for video conferencing to each of the two or more destination devices participating in the video conference, A grouping means that groups together multiple destination devices that receive audiovisual data from the aforementioned streaming distribution means to generate multiple groups, and Equipped with, The video conferencing relay means, with respect to each of the multiple groups generated by the grouping by the grouping means, transmits to each of the two or more destination devices, the video conferencing audiovisual data generated by synthesizing the audiovisual data received from each of the two or more destination devices, when two or more destination devices belonging to that group participate in the video conferencing. In response to a predetermined operation performed by a user of one of the receiving devices that is receiving a streaming broadcast, the receiving device will be made to participate in a video conference corresponding to the group to which the user belongs among the multiple groups. system.

2. The aforementioned streaming distribution means does not distribute audiovisual data to the destination device that is receiving audiovisual data from the aforementioned video conferencing relay means. The system according to claim 1.

3. The streaming distribution means resumes distributing audiovisual data to a destination device that has stopped receiving audiovisual data from the video conferencing relay means. The system according to claim 2.

4. The streaming distribution means distributes audiovisual data to destination devices that receive audiovisual data from the video conferencing relay means at a lower bitrate than it does to destination devices that do not receive audiovisual data from the video conferencing relay means. The system according to claim 1.

5. The video conferencing relay means receives audiovisual data from the source device and combines the audiovisual data received from the source device with the audiovisual data received from each of the two or more destination devices participating in the video conferencing to generate audiovisual data for the video conferencing. The system according to any one of claims 1 to 4.

6. The grouping means acquires attribute data indicating the attributes of each of the multiple destination devices that receive audiovisual data from the streaming distribution means, or the users of the destination devices, and performs grouping based on said attribute data. The system according to claim 1.

7. Attribute data receiving means that receives data input by the user of each of the multiple destination devices that receive audiovisual data from the aforementioned streaming distribution means as attribute data. Equipped with, The grouping means performs grouping based on the attribute data received by the attribute data receiving means. The system according to claim 6.

8. The grouping means acquires relationship data indicating the relationship between two or more distribution destination devices and performs grouping based on said relationship data. The system according to any one of claims 1 to 7.

9. The grouping means further groups the multiple destination devices belonging to each of the multiple groups generated by the grouping to generate multiple subgroups, The video conferencing relay means transmits to each of the two or more distribution destination devices, with respect to each of the multiple subgroups generated by the grouping by the grouping means, video conferencing audiovisual data generated by synthesizing the audiovisual data received from each of the two or more distribution destination devices belonging to the subgroup. The system according to any one of claims 1 to 8.

10. The streaming distribution means transmits display instruction data to each of the multiple destination devices, which instructs the display of a web page containing images represented by audiovisual data and images of operators for participating in a video conference. When an operation is performed on the operator for participating in the video conference, the video conferencing relay means transmits display instruction data to the destination device instructing it to display a web page containing images represented by audiovisual data for the video conference. The system according to claim 1.

11. The display instruction data transmitted by the streaming distribution means instructs the termination of the display of the web page being shown in accordance with the display instruction data when an operation is performed on the operator for joining the video conference. The system according to claim 10.

12. The display instruction data transmitted by the aforementioned video conferencing relay means instructs the display of a web page containing an image of an operator for instructing the termination of participation in the video conference, and when an operation is performed on said operator, it instructs the termination of the display of said web page. When an operation is performed on the operator for instructing the termination of participation in the video conference, the streaming distribution means transmits display instruction data to the destination device instructing it to display a web page that includes the image represented by the audiovisual data received from the distribution source device and the image of the operator for joining the video conference. The system according to claim 11.

13. The aforementioned video conferencing relay means receives data indicating a message from the source device and transmits the received data to the destination devices participating in the video conferencing. The system according to claim 1.

14. When the predetermined video conference time ends, the transmission of audiovisual data for the video conference by the video conference relay means is stopped, and the distribution of audiovisual data by the streaming distribution means is resumed. The system according to claim 1.

15. A streaming distribution means that receives audiovisual data from a source device and streams the audiovisual data to multiple destination devices, A video conferencing relay means that relays a video conference by receiving audiovisual data from each of two or more destination devices participating in a video conference, which are among a plurality of destination devices receiving audiovisual data from the streaming distribution means, receiving audiovisual data representing images captured by cameras connected to the destination devices and sound picked up by microphones connected to the destination devices, synthesizing the received audiovisual data to generate audiovisual data for video conferencing, and transmitting the generated audiovisual data for video conferencing to each of the two or more destination devices participating in the video conference, Equipped with, The status of users of the multiple destination devices is aggregated, and data showing the results of the aggregation is transmitted to the source device. system.

16. A streaming distribution means that receives audiovisual data from a source device and streams the audiovisual data to multiple destination devices, A video conferencing relay means that relays a video conference by receiving audiovisual data from each of two or more destination devices participating in a video conference, which are among a plurality of destination devices receiving audiovisual data from the aforementioned streaming distribution means, receiving audiovisual data representing images captured by cameras connected to the destination devices and sound picked up by microphones connected to the destination devices, synthesizing the received audiovisual data to generate audiovisual data for video conferencing, and transmitting the generated audiovisual data for video conferencing to each of the two or more destination devices participating in the video conference, A grouping means that groups together multiple destination devices that receive audiovisual data from the aforementioned streaming distribution means to generate multiple groups, and Equipped with, The video conferencing relay means, with respect to each of the multiple groups generated by the grouping by the grouping means, transmits to each of the two or more destination devices, the video conferencing audiovisual data generated by synthesizing the audiovisual data received from each of the two or more destination devices, when two or more destination devices belonging to that group participate in the video conferencing. With respect to any of the aforementioned groups, data indicating the user status of distribution destination devices belonging to that group that are not participating in the video conference is acquired, and this data is transmitted to distribution destination devices belonging to that group that are participating in the video conference. system.

17. A streaming distribution means that receives audiovisual data from a source device and streams the audiovisual data to multiple destination devices, A video conferencing relay means that relays a video conference by receiving audiovisual data from each of two or more destination devices participating in a video conference, which are among a plurality of destination devices receiving audiovisual data from the streaming distribution means, receiving audiovisual data representing images captured by cameras connected to the destination devices and sound picked up by microphones connected to the destination devices, synthesizing the received audiovisual data to generate audiovisual data for video conferencing, and transmitting the generated audiovisual data for video conferencing to each of the two or more destination devices participating in the video conference, Equipped with, With respect to any of the aforementioned multiple destination devices, the system records data indicating the time period during which audiovisual data was transmitted by the streaming distribution means and the time period during which audiovisual data was transmitted by the video conferencing relay means, and transmits the data to the destination device. system.

18. The audiovisual data transmitted by the streaming distribution means is stored, and, with respect to any of the multiple destination devices, the stored audiovisual data is transmitted to the destination device in response to a request from that destination device. The system according to claim 17.

19. A streaming distribution means that receives audiovisual data from a source device and streams the audiovisual data to multiple destination devices, A video conferencing relay means that relays a video conference by receiving audiovisual data from each of two or more destination devices participating in a video conference, which are among a plurality of destination devices receiving audiovisual data from the streaming distribution means, receiving audiovisual data representing images captured by cameras connected to the destination devices and sound picked up by microphones connected to the destination devices, synthesizing the received audiovisual data to generate audiovisual data for video conferencing, and transmitting the generated audiovisual data for video conferencing to each of the two or more destination devices participating in the video conference, Equipped with, The aforementioned streaming distribution means does not distribute audiovisual data to the destination device that is receiving audiovisual data from the aforementioned video conferencing relay means. The streaming distribution means resumes distributing audiovisual data to a destination device that has stopped receiving audiovisual data from the video conferencing relay means. The system stores the audiovisual data transmitted by the streaming distribution means, and if the stored audiovisual data has not yet been distributed, it distributes the undistributed audiovisual data to the distribution destination device that has stopped receiving audiovisual data from the video conferencing relay means. system.

20. A streaming distribution means that receives audiovisual data from a source device and streams the audiovisual data to multiple destination devices, A video conferencing relay means that relays a video conference by receiving audiovisual data from each of two or more destination devices participating in a video conference, which are among a plurality of destination devices receiving audiovisual data from the aforementioned streaming distribution means, receiving audiovisual data representing images captured by cameras connected to the destination devices and sound picked up by microphones connected to the destination devices, synthesizing the received audiovisual data to generate audiovisual data for video conferencing, and transmitting the generated audiovisual data for video conferencing to each of the two or more destination devices participating in the video conference, A grouping means that groups together multiple destination devices that receive audiovisual data from the aforementioned streaming distribution means to generate multiple groups, and Equipped with, The video conferencing relay means, with respect to each of the multiple groups generated by the grouping by the grouping means, transmits to each of the two or more destination devices, the video conferencing audiovisual data generated by synthesizing the audiovisual data received from each of the two or more destination devices, when two or more destination devices belonging to that group participate in the video conferencing. The grouping means further groups the multiple destination devices belonging to each of the multiple groups generated by the grouping to generate multiple subgroups, The video conferencing relay means transmits to each of the two or more distribution destination devices the audiovisual data for video conferencing generated by synthesizing the audiovisual data received from each of the two or more distribution destination devices belonging to each of the multiple subgroups generated by the grouping by the grouping means. The grouping means generates a subgroup that includes other users' destination devices selected by the user of any of the destination devices participating in the video conference, in response to a request from that destination device. system.

21. A streaming distribution means that receives audiovisual data from a source device and streams the audiovisual data to multiple destination devices, A video conferencing relay means that relays a video conference by receiving audiovisual data from each of two or more destination devices participating in a video conference, which are among a plurality of destination devices receiving audiovisual data from the aforementioned streaming distribution means, receiving audiovisual data representing images captured by cameras connected to the destination devices and sound picked up by microphones connected to the destination devices, synthesizing the received audiovisual data to generate audiovisual data for video conferencing, and transmitting the generated audiovisual data for video conferencing to each of the two or more destination devices participating in the video conference, A grouping means that groups together multiple destination devices that receive audiovisual data from the aforementioned streaming distribution means to generate multiple groups, and Equipped with, The video conferencing relay means, with respect to each of the multiple groups generated by the grouping by the grouping means, transmits to each of the two or more destination devices, the video conferencing audiovisual data generated by synthesizing the audiovisual data received from each of the two or more destination devices, when two or more destination devices belonging to that group participate in the video conferencing. The grouping means acquires attribute data indicating the attributes of each of the multiple destination devices that receive audiovisual data from the streaming distribution means, and performs grouping based on the attribute data. The aforementioned attribute data includes data indicating whether or not participation in the video conference is permitted. The grouping means performs grouping based on the attribute data, targeting only users who are able to participate in the video conference. system.

22. A streaming distribution means that receives audiovisual data from a source device and streams the audiovisual data to multiple destination devices, A video conferencing relay means that relays a video conference by receiving audiovisual data from each of two or more destination devices participating in a video conference, which are among a plurality of destination devices receiving audiovisual data from the aforementioned streaming distribution means, receiving audiovisual data representing images captured by cameras connected to the destination devices and sound picked up by microphones connected to the destination devices, synthesizing the received audiovisual data to generate audiovisual data for video conferencing, and transmitting the generated audiovisual data for video conferencing to each of the two or more destination devices participating in the video conference, A grouping means that groups together multiple destination devices that receive audiovisual data from the aforementioned streaming distribution means to generate multiple groups, and Equipped with, The video conferencing relay means, with respect to each of the multiple groups generated by the grouping by the grouping means, transmits to each of the two or more destination devices, the video conferencing audiovisual data generated by synthesizing the audiovisual data received from each of the two or more destination devices, when two or more destination devices belonging to that group participate in the video conferencing. The grouping means acquires attribute data indicating the attributes of each of the multiple destination devices that receive audiovisual data from the streaming distribution means, and performs grouping based on the attribute data. The grouping means assigns a group determined based on at least one of the number of destination devices belonging to an existing group and the user attributes of the destination devices belonging to that group to the destination devices of users who become eligible to participate in the video conference after the grouping has been performed. system.

23. A streaming distribution means that receives audiovisual data from a source device and streams the audiovisual data to multiple destination devices, A video conferencing relay means that relays a video conference by receiving audiovisual data from each of two or more destination devices participating in a video conference, which are among a plurality of destination devices receiving audiovisual data from the aforementioned streaming distribution means, receiving audiovisual data representing images captured by cameras connected to the destination devices and sound picked up by microphones connected to the destination devices, synthesizing the received audiovisual data to generate audiovisual data for video conferencing, and transmitting the generated audiovisual data for video conferencing to each of the two or more destination devices participating in the video conference, A grouping means that groups together multiple destination devices that receive audiovisual data from the aforementioned streaming distribution means to generate multiple groups, and Equipped with, The video conferencing relay means, with respect to each of the multiple groups generated by the grouping by the grouping means, transmits to each of the two or more destination devices, the video conferencing audiovisual data generated by synthesizing the audiovisual data received from each of the two or more destination devices, when two or more destination devices belonging to that group participate in the video conferencing. The grouping means assigns a role to a user of any of the multiple distribution destination devices belonging to any of the multiple groups, The video conferencing relay means transmits audiovisual data for the video conferencing that indicates the roles assigned by the grouping means. system.

24. A streaming distribution means that receives audiovisual data from a source device and streams the audiovisual data to multiple destination devices, A video conferencing relay means that relays a video conference by receiving audiovisual data from each of two or more destination devices participating in a video conference, which are among a plurality of destination devices receiving audiovisual data from the aforementioned streaming distribution means, receiving audiovisual data representing images captured by cameras connected to the destination devices and sound picked up by microphones connected to the destination devices, synthesizing the received audiovisual data to generate audiovisual data for video conferencing, and transmitting the generated audiovisual data for video conferencing to each of the two or more destination devices participating in the video conference, A grouping means that groups together multiple destination devices that receive audiovisual data from the aforementioned streaming distribution means to generate multiple groups, and Equipped with, The video conferencing relay means, with respect to each of the multiple groups generated by the grouping by the grouping means, transmits to each of the two or more destination devices, the video conferencing audiovisual data generated by synthesizing the audiovisual data received from each of the two or more destination devices, when two or more destination devices belonging to that group participate in the video conferencing. The grouping means acquires attribute data indicating the attributes of each of the multiple destination devices that receive audiovisual data from the streaming distribution means, and performs grouping based on the attribute data. The attribute data includes data showing past participation history in video conferences. The grouping means performs grouping based on the data indicating the participation history, such that predetermined conditions regarding the participation history are met among multiple groups. system.

25. A streaming distribution means that receives audiovisual data from a source device and streams the audiovisual data to multiple destination devices, A video conferencing relay means that relays a video conference by receiving audiovisual data from each of two or more destination devices participating in a video conference, which are among a plurality of destination devices receiving audiovisual data from the aforementioned streaming distribution means, receiving audiovisual data representing images captured by cameras connected to the destination devices and sound picked up by microphones connected to the destination devices, synthesizing the received audiovisual data to generate audiovisual data for video conferencing, and transmitting the generated audiovisual data for video conferencing to each of the two or more destination devices participating in the video conference, A grouping means that groups together multiple destination devices that receive audiovisual data from the aforementioned streaming distribution means to generate multiple groups, and Equipped with, The video conferencing relay means, with respect to each of the multiple groups generated by the grouping by the grouping means, transmits to each of the two or more destination devices, the video conferencing audiovisual data generated by synthesizing the audiovisual data received from each of the two or more destination devices, when two or more destination devices belonging to that group participate in the video conferencing. After the video conference begins, the number of users participating in each of the aforementioned groups is counted, and based on this count, the group to which any user belongs is changed, and a notification is sent to the receiving device of the user whose group has been changed, prompting them to move to the new group's video conference. system.

26. A streaming distribution means that receives audiovisual data from a source device and streams the audiovisual data to multiple destination devices, A video conferencing relay means that relays a video conference by receiving audiovisual data from each of two or more destination devices participating in a video conference, which are among a plurality of destination devices receiving audiovisual data from the streaming distribution means, receiving audiovisual data representing images captured by cameras connected to the destination devices and sound picked up by microphones connected to the destination devices, synthesizing the received audiovisual data to generate audiovisual data for video conferencing, and transmitting the generated audiovisual data for video conferencing to each of the two or more destination devices participating in the video conference, Equipped with, The streaming distribution means transmits display instruction data to each of the multiple destination devices, which instructs the display of a web page containing images represented by audiovisual data and images of operators for participating in a video conference. When an operation is performed on the operator for participating in the video conference, the video conferencing relay means transmits display instruction data to the destination device instructing it to display a web page containing images represented by audiovisual data for the video conference. It stores multiple pre-prepared templates according to the user's attributes, The streaming distribution means and the video conferencing relay means generate the display instruction data using a template that corresponds to the user attributes of the destination device. system.

27. The aforementioned streaming distribution means converts the audio represented by the audiovisual data received from the distribution source device into text data, translates the text data into a language corresponding to the user attributes of the destination device, synthesizes audio representing the content of the translated text data, and streams the audiovisual data representing the synthesized audio to the destination device. The system according to claim 1.

28. Obtain attribute data indicating the user attributes of the destination device, The aforementioned video conferencing relay means generates audiovisual data for the video conference that displays the attributes indicated by the attribute data in association with images of users of the receiving devices participating in the video conference. The system according to claim 1.

29. A streaming distribution means that receives audiovisual data from a source device and streams the audiovisual data to multiple destination devices, A video conferencing relay means that relays a video conference by receiving audiovisual data from each of two or more destination devices participating in a video conference, which are among a plurality of destination devices receiving audiovisual data from the streaming distribution means, receiving audiovisual data representing images captured by cameras connected to the destination devices and sound picked up by microphones connected to the destination devices, synthesizing the received audiovisual data to generate audiovisual data for video conferencing, and transmitting the generated audiovisual data for video conferencing to each of the two or more destination devices participating in the video conference, Equipped with, The audiovisual data for the video conference generated by the aforementioned video conference relay means is transmitted to the distribution devices of observer users who are not participating in the video conference. system.

30. The aforementioned video conferencing relay means transmits data to destination devices participating in the video conference in which audiovisual data is being transmitted to the observer user's destination device, informing them that the data is being viewed by the observer user. The system according to claim 29.

31. A streaming distribution means that receives audiovisual data from a source device and streams the audiovisual data to multiple destination devices, A video conferencing relay means that relays a video conference by receiving audiovisual data from each of two or more destination devices participating in a video conference, which are among a plurality of destination devices receiving audiovisual data from the streaming distribution means, receiving audiovisual data representing images captured by cameras connected to the destination devices and sound picked up by microphones connected to the destination devices, synthesizing the received audiovisual data to generate audiovisual data for video conferencing, and transmitting the generated audiovisual data for video conferencing to each of the two or more destination devices participating in the video conference, Equipped with, The system identifies a first distribution destination device used by a user who wishes to receive a consultation, and a plurality of second distribution destination devices used by a user who will receive the consultation. In response to a request from the first distribution destination device, the video conferencing relay means relays the video conference between the first distribution destination device and the second distribution destination devices. system.

32. In response to a request from the second destination device, a third destination device used by a different advisor user than the advisor user using the second destination device is identified, and after relaying the video conference between the second destination device and the third destination device using the video conferencing relay means, the relay of the video conference between the first destination device and the second destination device is switched to relaying the video conference between the first destination device and the third destination device. The system according to claim 31.

33. The video conferencing relay means receives audiovisual data from the source device, and combines the audiovisual data received from the source device with the audiovisual data received from each of the two or more destination devices participating in the video conferencing to generate audiovisual data for the video conferencing. The aforementioned video conferencing relay means synthesizes audiovisual data representing the content selected by the user of the source device from among multiple distributed content to generate audiovisual data for the video conference. The system according to claim 1.

34. A streaming distribution means that receives audiovisual data from a source device and streams the audiovisual data to multiple destination devices, A video conferencing relay means that relays a video conference by receiving audiovisual data from each of two or more destination devices participating in a video conference, which are among a plurality of destination devices receiving audiovisual data from the streaming distribution means, receiving audiovisual data representing images captured by cameras connected to the destination devices and sound picked up by microphones connected to the destination devices, synthesizing the received audiovisual data to generate audiovisual data for video conferencing, and transmitting the generated audiovisual data for video conferencing to each of the two or more destination devices participating in the video conference, Equipped with, The streaming distribution means transmits display instruction data to each of the multiple destination devices, which instructs the display of a web page containing images represented by audiovisual data and images of operators for participating in a video conference. In response to instructions from the aforementioned distribution source device, two or more groups are merged into one new group. The aforementioned video conferencing relay means relays the video conference between distribution destination devices belonging to the new group. system.

35. A streaming distribution means that receives audiovisual data from a source device and streams the audiovisual data to multiple destination devices, A video conferencing relay means that relays a video conference by receiving audiovisual data from each of two or more destination devices participating in a video conference, which are among a plurality of destination devices receiving audiovisual data from the aforementioned streaming distribution means, receiving audiovisual data representing images captured by cameras connected to the destination devices and sound picked up by microphones connected to the destination devices, synthesizing the received audiovisual data to generate audiovisual data for video conferencing, and transmitting the generated audiovisual data for video conferencing to each of the two or more destination devices participating in the video conference, A grouping means that groups together multiple destination devices that receive audiovisual data from the aforementioned streaming distribution means to generate multiple groups, and Equipped with, The video conferencing relay means, with respect to each of the multiple groups generated by the grouping by the grouping means, transmits to each of the two or more destination devices, the video conferencing audiovisual data generated by synthesizing the audiovisual data received from each of the two or more destination devices, when two or more destination devices belonging to that group participate in the video conferencing. Two or more of the aforementioned groups, where at least one of the number of participants and their attributes satisfies a predetermined condition, are merged into one new group. The aforementioned video conferencing relay means relays the video conference between distribution destination devices belonging to the new group. system.

36. A streaming distribution means that receives audiovisual data from a source device and streams the audiovisual data to multiple destination devices, A video conferencing relay means that relays a video conference by receiving audiovisual data from each of two or more destination devices participating in a video conference, which are among a plurality of destination devices receiving audiovisual data from the aforementioned streaming distribution means, receiving audiovisual data representing images captured by cameras connected to the destination devices and sound picked up by microphones connected to the destination devices, synthesizing the received audiovisual data to generate audiovisual data for video conferencing, and transmitting the generated audiovisual data for video conferencing to each of the two or more destination devices participating in the video conference, A grouping means that groups together multiple destination devices that receive audiovisual data from the aforementioned streaming distribution means to generate multiple groups, and Equipped with, The video conferencing relay means, with respect to each of the multiple groups generated by the grouping by the grouping means, transmits to each of the two or more destination devices, the video conferencing audiovisual data generated by synthesizing the audiovisual data received from each of the two or more destination devices, when two or more destination devices belonging to that group participate in the video conferencing. Status data is obtained indicating on-site participants who have not received audiovisual data from the aforementioned streaming distribution means. The grouping means performs grouping not only on users of the destination device receiving audiovisual data from the streaming distribution means, but also on-site participants indicated by the status data. system.

37. A streaming distribution means that receives audiovisual data from a source device and streams the audiovisual data to multiple destination devices, A video conferencing relay means that relays a video conference by receiving audiovisual data from each of two or more destination devices participating in a video conference, which are among a plurality of destination devices receiving audiovisual data from the streaming distribution means, receiving audiovisual data representing images captured by cameras connected to the destination devices and sound picked up by microphones connected to the destination devices, synthesizing the received audiovisual data to generate audiovisual data for video conferencing, and transmitting the generated audiovisual data for video conferencing to each of the two or more destination devices participating in the video conference, Equipped with, The streaming distribution means transmits display instruction data to each of the multiple destination devices, which instructs the display of a web page containing images represented by audiovisual data and images of operators for participating in a video conference. In response to a request from any of the destination devices, data indicating the attributes of other users belonging to the same group as the user of that destination device is sent to that destination device. system.

38. The grouping means performs grouping based on the attribute data, targeting only users who meet predetermined conditions. The system according to claim 6.

39. A streaming distribution means that receives audiovisual data from a source device and streams the audiovisual data to multiple destination devices, A video conferencing relay means that relays a video conference by receiving audiovisual data from each of two or more destination devices participating in a video conference, which are among a plurality of destination devices receiving audiovisual data from the aforementioned streaming distribution means, receiving audiovisual data representing images captured by cameras connected to the destination devices and sound picked up by microphones connected to the destination devices, synthesizing the received audiovisual data to generate audiovisual data for video conferencing, and transmitting the generated audiovisual data for video conferencing to each of the two or more destination devices participating in the video conference, A grouping means that groups together multiple destination devices that receive audiovisual data from the aforementioned streaming distribution means to generate multiple groups, and Equipped with, The video conferencing relay means, with respect to each of the multiple groups generated by the grouping by the grouping means, transmits to each of the two or more destination devices, the video conferencing audiovisual data generated by synthesizing the audiovisual data received from each of the two or more destination devices, when two or more destination devices belonging to that group participate in the video conferencing. The grouping means acquires attribute data indicating the attributes of each of the multiple destination devices that receive audiovisual data from the streaming distribution means, and performs grouping based on the attribute data. The message sent from the source device is sent only to the destination devices of users who are participating in any of the multiple groups and whose attributes, as indicated by the attribute data, meet predetermined conditions. system.

40. A streaming distribution means that receives audiovisual data from a source device and streams the audiovisual data to multiple destination devices, A video conferencing relay means that relays a video conference by receiving audiovisual data from each of two or more destination devices participating in a video conference, which are among a plurality of destination devices receiving audiovisual data from the aforementioned streaming distribution means, receiving audiovisual data representing images captured by cameras connected to the destination devices and sound picked up by microphones connected to the destination devices, synthesizing the received audiovisual data to generate audiovisual data for video conferencing, and transmitting the generated audiovisual data for video conferencing to each of the two or more destination devices participating in the video conference, A grouping means that groups together multiple destination devices that receive audiovisual data from the aforementioned streaming distribution means to generate multiple groups, and Equipped with, The video conferencing relay means, with respect to each of the multiple groups generated by the grouping by the grouping means, transmits to each of the two or more destination devices, the video conferencing audiovisual data generated by synthesizing the audiovisual data received from each of the two or more destination devices, when two or more destination devices belonging to that group participate in the video conferencing. The grouping means acquires attribute data indicating the attributes of each of the multiple destination devices that receive audiovisual data from the streaming distribution means, and performs grouping based on the attribute data. Only on the destination devices of users whose attributes, as indicated by the attribute data, meet predetermined conditions, a message input field for the source device is displayed, and the message entered in that input field is sent to the source device. system.

Images