Online distribution system

Abstract

To provide novel technique for making a possible for having an individual sound communication only with a specific participant in an online event where a plurality of participants joins.SOLUTION: An online distribution system sets an individual room by one part of a plurality of guests and two or more participants into a whole room that all of the plurality of guests is a participant. Sound received from a host terminal is transmitted to a guest terminal of the participant of the whole room. The sound received from the guest terminal of the participant in the individual room is transmitted to the guest terminal of the other participant in the individual room, and is not transmitted to the guest terminal other than the participant in the individual room. A method for setting the individual room contains steps of: receiving a room participant request using a code optionally corresponded to the individual room from the guest terminal of a first guest from a plurality of guests; and making the first guest participated in the individual room.SELECTED DRAWING: Figure 6

Description

[Technical field]

[0001] The present invention relates to a technique for distributing video and audio over a network. [Background technology]

[0002] There are known technologies that realize real-time communication using video and audio among multiple terminals connected by a network. Such technologies are applied to various web services such as online conferences and live video distribution. For example, Patent Document 1 proposes an online conference system that connects a server and multiple terminals via the Internet and enables mutual transmission and reception of video and audio between the multiple terminals. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2021-184189 A Summary of the Invention [Problem to be solved by the invention]

[0004] In general online conference systems such as Zoom and Microsoft Teams, the speaker's voice is uniformly distributed to all participants in the conference. Therefore, for example, even if you want to participate in a conference and have certain people talk to you without the other participants hearing it, this cannot be achieved with the functions of general online conference systems. Although there is a breakout room function that allows individual conferences to be held only with some participants, when you enter a breakout room you cannot hear the entire conference voice (in other words, you are removed from the general conference), so it cannot be used for the above purposes.

[0005] In light of such problems, the aforementioned Patent Document 1 proposes an idea of ​​a direct talk function that enables direct conversations only between specific terminals during an online conference. By using this direct talk function, for example, it becomes possible to have a private conversation with a specific participant while listening to the entire audio of the online conference. Patent Document 1 proposes two methods for starting a direct talk: a nomination request and a group request. The nomination request is a method for selecting a person to talk to from among the images of participants displayed on the screen of the online conference. The group request is a method for starting a direct talk by registering a combination of members who will be involved in the direct talk as a group in advance and selecting the group during the online conference.

[0006] The method of Patent Document 1 is highly convenient for closed meetings where participants know each other. In closed meetings, participants are expected to show their faces or use their real names, so it is possible to identify the person to talk to on the screen, and it is easy to set up and register groups in advance. However, for open online events such as public online seminars, online tours, and live video distribution, the following problems are of concern.

[0007] First, from the perspective of protecting personal information and privacy, it is often not possible to display the faces or real names of participants on the screen. In such cases, even if it is possible to display a list of participants, it will only display nicknames and avatars, not information that can identify individuals. This makes it difficult to select a talk partner using a method such as the designated request in Patent Document 1. Even if you only look at the nickname and avatar, it is difficult to guarantee that the person you want to talk to is the correct person. This is because they are likely to be hesitant to send a nomination request because they do not have the necessary skills.

[0008] Secondly, in the case of open online events, the number of participants can exceed tens or even hundreds. When the number of participants is large, it is difficult to display a list of participants on the screen, and even if it were possible to display them, it would be extremely difficult to operate and convenient to find the desired person among a huge number of participants.

[0009] Thirdly, in open events where various participants gather, it is difficult to predict what combinations will be used for direct talks, so it is difficult to prepare and register groups in advance. Also, as the number of participants increases, the number of combination patterns also increases, and the number of groups becomes enormous, so it is not realistic to display a list of groups on a screen or to select a desired group from a huge number of groups.

[0010] The present invention has been made in consideration of the above-mentioned situation, and its purpose is to provide a new technology that enables individual voice communication with specific participants in an online event attended by multiple participants. [Means for solving the problem]

[0011] In order to achieve the above-mentioned object, the online distribution system of the present invention is an online distribution system for distributing video and audio via a network, and has a server capable of communicating via the network with a host terminal operated by a host and multiple guest terminals operated by each of multiple guests, and the server performs the following operations: setting a general room with all of the multiple guests as participants; setting individual rooms in the general room with two or more participants among the multiple guests; uniformly transmitting audio received from the host terminal to the guest terminals of the participants in the general room; and transmitting audio received from the guest terminal of the participant in the individual room to the guest terminals of other participants in the individual room, but not to the guest terminals of guests other than the participants in the individual room, and setting the individual room includes accepting a room participation request using a code uniquely associated with the individual room from the guest terminal of a first guest among the multiple guests, and having the first guest participate in the individual room.

[0012] Setting the individual room may include accepting a room creation request from a guest terminal of a second guest among the plurality of guests, generating a new individual room in response to the room creation request and having the second guest participate in the new individual room, issuing a code to be used when participating in the new individual room, and notifying the guest terminal of the second guest of the code.

[0013] Accepting a room participation request using the code from the guest terminal of the first guest may include displaying a code input screen on the guest terminal of the first guest, receiving the code entered on the input screen from the guest terminal of the first guest, and identifying an individual room to which the first guest requests participation based on the code received from the guest terminal of the first guest.

[0014] The code may be a URL of the individual room, and accepting a room participation request using the code from a guest terminal of the first guest may include detecting a terminal that has accessed the URL, and if the terminal that has accessed the URL is the same as the guest terminal of the first guest, determining that the request is a room participation request to the individual room from the first guest.

[0015] The server may further perform the following: setting up a host call room with some of the multiple guests and the host as participants; and transmitting audio received from a guest terminal of a guest participating in the host call room to a guest terminal of a guest other than the participants in the host call room.

[0016] The server may further perform the following: setting up a host call room with some of the multiple guests and the host as participants; allowing the host or a guest participating in the host call room to select whether or not to make the call in the host call room public; when making the call in the host call room public is selected, transmitting audio received from a guest terminal of a guest participating in the host call room to a guest terminal of a guest other than the participants in the host call room; and when making the call in the host call room private is selected, controlling so that the audio received from the host terminal and the audio received from the guest terminal of a guest participating in the host call room are not transmitted to the guest terminal of a guest other than the participants in the host call room.

[0017] The present invention can be understood as an online distribution system or online event system having at least a part of the configurations described above or described below, or as a server or terminal constituting a part of these systems. The present invention may also be understood as an online distribution method or online event method having at least a part of the processes described above or described below, or as a method for creating an individual room or a method for entering an individual room. The present invention may also be understood as a program for causing a computer to function as these systems, a program for causing a computer to execute these methods, or a computer-readable storage medium non-temporarily storing such a program. Effect of the Invention

[0018] According to the present invention, in an online event attended by a plurality of participants, an environment (individual room) in which voice communication can be carried out individually with specific participants can be used with simple operations. [Brief description of the drawings]

[0019] [Figure 1] FIG. 1 shows an overview of the online distribution system. [Diagram 2] Figure 2 is an example of the top screen of an online tour. [Diagram 3] Figure 3 is an example of a tour screen displayed on a guest's terminal while participating in an online tour. [Figure 4] FIG. 4 is a diagram showing a schematic diagram of distribution control of the voice of the host and the voice of the guest in the online distribution system. [Diagram 5] FIG. 5 is a diagram showing an example of a screen displayed when starting an individual call (setting an individual room). [Figure 6] FIG. 6 is a diagram showing a procedure for starting an individual call (setting an individual room). [Figure 7] FIG. 7 is a diagram showing an example of a screen displayed when starting an individual call (setting an individual room). [Figure 8]FIG. 8 is a diagram showing a procedure for starting an individual call (setting an individual room). DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0020] The present invention relates to an online distribution system that distributes video and audio through a network, and is applicable to various online events that use the Internet, such as online conferences, online seminars, online classes, live video distribution, online tours, and metaverses. Online events can be broadly divided into closed events ( There are two types of events: online events (for example, an internal company meeting, an online school class, etc.) and open events in which people who do not know each other can participate (for example, a public seminar, a live broadcast, an online tour, etc.). The present invention can be preferably applied to both types of events, but is more preferably applied to open events. Therefore, below, a specific application example of the present invention will be described using an online tour as an example of an open event.

[0021] <System configuration> FIG. 1 is a diagram showing an overall view of an online distribution system 1. As shown in FIG.

[0022] The online distribution system 1 includes a server 10, a host terminal 11 operated by a host, and a plurality of guest terminals 12 operated by a plurality of guests. The server 10, the host terminal 11, and the guest terminal 12 can transmit and receive data to and from each other via the Internet. Here, the "host" refers to the organizer of the online event or the organizer's staff, and in the case of an online tour, which is the use case of this embodiment, the tour guide corresponds to this. Furthermore, the "guest" refers to the participant of the online event, and in the case of an online tour, the tour participant corresponds to this. This online tour is an open event that anyone can participate in by applying, and the participants in the event include people who know each other (for example, friends who applied together), but may also include people who have never met each other.

[0023] The server 10 is a device that provides tour participants with a simulated travel experience using video and audio. The server 10 can be configured, for example, by a general-purpose computer system equipped with a processor (CPU), memory, storage, communication devices, etc., and the functions of the server 10 described below are realized by the processor executing a program deployed from the storage to the memory. The server 10 may be configured by a single computer, or may be configured by multiple computers working together as in distributed computing, or may be dynamically allocated resources as in cloud computing.

[0024] The host terminal 11 is a terminal device used by a tour guide. The host terminal 11 can be configured, for example, by a general-purpose computer equipped with a processor (CPU), memory, storage, communication device, display device, input device, microphone, speaker, etc. The guest terminal 12 is a terminal device used by a guest participating in a tour. The guest terminal 12 can also be configured, for example, by a general-purpose computer equipped with a processor (CPU), memory, storage, communication device, display device, input device, microphone, speaker, etc. As the host terminal 11 and the guest terminal 12, for example, a personal computer, a tablet terminal, a smartphone, a game device, smart glasses, a VR terminal, an XR terminal, etc. can be used. The functions of the host terminal 11 and the guest terminal 12 are realized by a pre-installed native app or a web app that can be used on a web browser. The native app can be downloaded from the server 10 or an app store.

[0025] <Online Tour> 2 and 3, an overview of the online tour provided by the online distribution system 1 will be described. An online tour is a web service that provides a simulated travel experience without actually visiting tourist spots by allowing participants to watch images of tourist spots and audio of tour guides on their devices using video content, VR / XR content, or live streaming from the tourist spots.

[0026] FIG. 2 is an example of the top screen of an online tour. When a person who wishes to participate in an online tour accesses the online tour site using a browser or a dedicated application from his / her guest terminal 12, the top screen of FIG. 2 is displayed. When you select this, a login screen (not shown) will be displayed, and you will need to authenticate yourself with your ID and password before logging in. If you are a new user, you will need to enter your personal information (name, address, etc.), a nickname to be used on the tour, a payment method (credit card, etc.) and other information on a designated registration screen (not shown) to register as a user.

[0027] On the top screen of FIG. 2, a list of multiple tours is displayed in tiles. The tile representing each tour displays the tour name, destination, date and time of departure, thumbnail images of tourist attractions, and the like. By selecting a desired tour from this list, the guest can participate in or make a reservation for participation in that tour. In other words, if a tour that is currently being conducted is selected, the guest terminal 12 is connected to a video session of that tour, and video and audio of that tour are transmitted to the guest terminal 12. On the other hand, if a tour that has not yet started is selected, the guest can check the details of that tour and make a reservation for participation in that tour.

[0028] 3 shows an example of a tour screen displayed on the guest terminal 12 during participation in an online tour. The tour screen is made up of a main window 30, a sub-window 31, a main menu 32, and a location menu 33.

[0029] The main window 30 is an area in which the tour video is displayed. In this embodiment, video of XR content captured by a 360-degree camera is displayed in the main window 30. The sub-window 31 is an area in which the video of the tour guide (host) and information on individual calls (described later) are displayed. The sub-window 31 is displayed superimposed on the main window 30. The display position and display size of the sub-window 31 may be changed, and it may be possible to switch between display and non-display.

[0030] The main menu 32 includes an individual call button 320, a guide synchronization button 321, a cart button 322, a user button 323, a help button 324, an exit button 325, and the like. The individual call button 320 is used when starting an individual call with another guest (details will be described later). The guide synchronization button 321 is a button for switching whether or not to synchronize the image in the main window 30 with the viewpoint image displayed on the host terminal 11 of the tour guide. The tour guide performs an explanation in accordance with the displayed image while operating the pan / tilt / zoom of the image on the host terminal 11. When the guide synchronization button 321 is set to "synchronization," the same viewpoint image as that of the host terminal 11 is displayed on the main window 30 of the guest terminal 12, and the guest can listen to the guide while watching the same image as the tour guide. On the other hand, when the guide synchronization button 321 is set to "asynchronous," the guest can use an input device such as a mouse or a touch panel to perform operations such as enlarging / reducing the image displayed on the main window 30 and changing the viewpoint. Furthermore, if the guest is wearing a head mounted display, the viewpoint of the video is controlled to change in conjunction with the movement of the guest's head and gaze. Pressing the cart button 322 transitions to a purchase screen for souvenirs being sold in the XR content. The user button 323 is used to confirm or change the user's properties. Pressing the help button 324 displays instructions on how to use the tour screen, Q&A, etc. Pressing the exit button 325 exits the tour.

[0031] The location menu 33 is a menu that displays a list of tourist spots and points of interest included in the XR content. When a guest selects an item from the location menu 33, the image in the main window 30 is switched to the viewpoint image of that item, or an explanation of the item is superimposed on the main window 30.

[0032] <Individual call> 4 to 6, the individual call function of the online distribution system 1 will be described. The individual call function is a function for making a call to some of the guests who are participating in the online tour. This function allows two-way voice communication between guests individually while watching the tour video and the tour guide's audio. The audio of the individual call is not distributed to other guests. This function is used, for example, for private conversations between friends who have joined a tour together.

[0033] Fig. 4 shows a schematic diagram of distribution control of the voice of the host and the voice of the guest in the online distribution system 1. Fig. 4 shows an example in which a host 40 and three guests 41 to 43 are participating in an online tour. Note that for the sake of convenience, only three participants are shown, but in actual operation, the number of participants may range from several to several tens of people, or even more than a hundred people.

[0034] As shown in FIG. 4A, the online tour can be considered as one room R1 (conference room) in which the host 40 and all the guests 41 to 43 are participants. Here, the room R1 consisting of all the participants of the online tour is referred to as the "general room." In the general room R1, the server 10 receives the voice of the host 40 from the host terminal 11, and performs control to uniformly transmit the voice to the guest terminals 12 of all the guests 41 to 43 participating in the general room R1. This allows the guests 41 to 43 participating in the online tour to hear the guide voice of the host 40. Note that, in order to reduce traffic, the server 10 does not need to receive the voice of the guest terminals 12 in the general room R1.

[0035] FIG. 4B shows distribution control when an individual call is made between the guest 41 and the guest 42. The individual call between the guest 41 and the guest 42 can be realized by setting an individual room R2 in which only the guest 41 and the guest 42 participate in the general room R1. In the individual room R2, the server 10 transmits the voice received from the guest terminal 12 of the guest 41 to the guest terminal 12 of the other guest 42, and conversely transmits the voice received from the guest terminal 12 of the guest 42 to the guest terminal 12 of the guest 41. At this time, the server 10 does not transmit the voice received from the participants (guests 41 and 42) of the individual room R2 to the guest terminal 12 of the guest 43 who is not participating in the individual room R2. By such distribution control, two-way voice communication can be performed only between the guest 41 and the guest 42 who participate in the individual room R2.

[0036] The online distribution system 1 can be constructed, for example, by OME (OvenMediaEngine) based on WebRTC SFU (Web Real-Time Communication Selective Forwarding Unit). In addition, individual calls between guest terminals can be constructed using open source such as LiveKit.

[0037] Fig. 5 shows an example of a screen when starting an individual call (setting an individual room), and Fig. 6 shows the procedure when starting an individual call (setting an individual room). The procedure in which a guest 42 (a second guest) creates a new individual room and allows a guest 41 (a first guest) to participate therein will be described.

[0038] First, the guest 42 presses the individual call button 320 on the main menu 32 of his / her guest terminal 12 and selects “Create new individual call” (step S60). Then, a room creation request is sent from the guest terminal 12 of the guest 42 to the server 10.

[0039] When the server 10 accepts the room creation request, it creates a new individual room (step S61) and adds the guest 42 to the individual room (step S62). Furthermore, the server 10 issues an authentication passcode for participating in the individual room. This passcode is uniquely (one-to-one) associated with the individual room. The server 10 notifies the guest terminal 12 of the guest 42 of the issued passcode (step S63). Then, a screen 50 showing the passcode is displayed on the guest terminal 12 of the guest 42 (step S64).

[0040] The guest 42 transmits the passcode to the person with whom he / she wishes to make a private call (guest 41 in this example). The passcode can be transmitted by any means, such as email, short message, communication tool, SNS, or telephone.

[0041] Next, the guest 41 presses the individual call button 320 from the main menu 32 of his / her guest terminal 12 and selects "Join individual call" (step S65). Then, a passcode input screen 51 is displayed on the guest terminal 12 of the guest 41. When the guest 41 inputs the passcode transmitted by the guest 42 into the input screen 51, the information is transmitted to the server 10 together with a room participation request (step S66).

[0042] When the server 10 receives the room participation request together with the passcode from the guest terminal 12 of the guest 41, the server 10 identifies the individual room to which the guest 41 requests participation based on the passcode (step S67). Then, if an individual room associated with the passcode is found, the server 10 adds the guest 41 to that individual room (step S68).

[0043] By the above-mentioned procedure, an individual room in which the guest 41 and the guest 42 participate can be easily set. According to this method, only guests who know the passcode can participate in the individual room, so there is no risk of an unintended person entering the individual room. In addition, with this method, there is no need to display a list of participants on the screen, or to display personal information such as faces and real names on the screen, and there is no need to register groups for individual calls in advance. Therefore, there is an advantage that it is easy to operate even for open online events such as online tours.

[0044] <Other examples> Another example of the procedure for starting an individual call will be described with reference to Fig. 7 and Fig. 8. Fig. 7 shows an example of a screen when starting an individual call (setting an individual room), and Fig. 8 shows the procedure for starting an individual call (setting an individual room). The procedure for a guest 42 (second guest) to create a new individual room and have a guest 41 (first guest) participate therein will be described.

[0045] First, the guest 42 presses the individual call button 320 on the main menu 32 of his / her guest terminal 12 and selects "Create new individual call" (step S80). Then, a room creation request is sent from the guest terminal 12 of the guest 42 to the server 10.

[0046] When the server 10 receives the room creation request, it creates a new individual room (step S81) and adds the guest 42 to the individual room (step S82). Furthermore, the server 10 issues a URL for participating in the individual room. This URL is uniquely (one-to-one) associated with the individual room. The server 10 notifies the guest terminal 12 of the guest 42 of the issued URL (step S83). Then, a screen 70 showing the URL is displayed on the guest terminal 12 of the guest 42 (step S84).

[0047] The guest 42 transmits the URL to the person with whom he / she wishes to make a private call (guest 41 in this example). The URL can be transmitted by any means, such as email, short message, communication tool, SNS, or telephone.

[0048] Next, the guest 41 uses his / her guest terminal 12 to access the URL transmitted from the guest 42 (step S85). In this example, accessing the URL of this individual room is a request to participate in the individual room.

[0049] When the server 10 detects a terminal that has accessed the URL of an individual room, it checks whether the detected terminal is included among the guest terminals 12 of the tour participants, and The server 10 identifies the URL of the individual room (step S86). For example, by checking whether a session, a cookie, a token, an IP address, etc., matches, the server 10 can determine that the guest terminal 12 of the guest 41 has accessed the URL of the individual room. Then, the server 10 adds the guest 41 to the individual room (step S87).

[0050] The above procedure also makes it easy to set up an individual room in which guests 41 and 42 participate. With this method, only guests who know the URL can participate in the individual room, so there is no risk of an unintended person entering the individual room. In addition, with this method, there is no need to display a list of participants on the screen, or to display personal information such as faces and real names on the screen, and there is no need to register groups for individual calls in advance. Therefore, there is an advantage that it is easy to operate even for open online events such as online tours.

[0051] (others) The above embodiment merely describes one application example of the present invention, and the scope of the present invention is not limited to the above embodiment. For example, the above embodiment describes an example in which the present invention is applied to an online tour, but the present invention can also be applied to other online events. In addition, in the above embodiment, a passcode and a URL are exemplified as a code for identifying an individual room, but a QR code (registered trademark), a voice code, etc. may also be used. [Explanation of symbols]

[0052] 1: Online distribution system 10: Server 11: Host terminal 12: Guest terminal 30: Main window 31: Sub window 32:Main Menu 33: Places Menu 40: Host 41, 42, 43: Guest 50: Passcode screen 51: Passcode entry screen 70: Screen showing URL 320: Individual call button 321: Guide Sync button 322: Cart button 323: User button 324: Help button 325: Exit button R1: Whole room R2: Private room

Claims

1. An online distribution system for distributing video and audio over a network, comprising: The system includes a server capable of communicating with a host terminal operated by a host and a plurality of guest terminals operated by a plurality of guests via a network, The server, Setting a general room with all of the guests as participants; Setting individual rooms for two or more participants who are part of the plurality of guests in the general room; Transmitting the voice received from the host terminal to all guest terminals of the participants in the general room; transmitting the voice received from the guest terminal of the participant in the individual room to the guest terminal of another participant in the individual room, and not transmitting the voice to the guest terminal of a guest other than the participant in the individual room; Setting the individual room includes: receiving a room participation request using a code uniquely associated with the individual room from a guest terminal of a first guest among the plurality of guests; and joining the first guest to the individual room. Online distribution system.

2. Setting the individual room includes: receiving a room creation request from a guest terminal of a second guest among the plurality of guests; generating a new individual room in response to the room creation request and having the second guest participate in the new individual room; issuing a code to be used when participating in the new individual room, and notifying a guest terminal of the second guest of the code; The online distribution system according to claim 1 .

3. Receiving a room participation request using the code from the guest terminal of the first guest, Displaying a code input screen on the guest terminal of the first guest; receiving the code entered on the input screen from a guest terminal of the first guest; and identifying an individual room in which the first guest requests to join based on the code received from the guest terminal of the first guest. The online distribution system according to claim 1 .

4. The code is a URL of the individual room, Receiving a room participation request using the code from the guest terminal of the first guest, Detecting a terminal that has accessed the URL; and determining, when the terminal accessing the URL is the same as the guest terminal of the first guest, that the request is a room participation request from the first guest to the individual room. The online distribution system according to claim 1 .

5. The server, A host call room in which some of the guests and the host are participants. Setting the and transmitting the voice received from the guest terminal of the guest participating in the host call room to the guest terminal of the guest other than the participants in the host call room. The online distribution system according to claim 1 .

6. The server, setting a host call room with some of the guests and the host as participants; Providing the host or a guest participating in the host call room with the option of whether or not to make the call in the host call room public; and when it is selected to make the call in the host call room public, control is further executed to transmit audio received from a guest terminal of a guest participating in the host call room to a guest terminal of a guest other than the participants in the host call room, and when it is selected to make the call in the host call room private, control is further executed to prevent the audio received from the host terminal and the audio received from the guest terminal of a guest participating in the host call room from being transmitted to the guest terminal of a guest other than the participants in the host call room. The online distribution system according to claim 1 .

Images