Online distribution system and online distribution method

Abstract

To provide a new technology suitable for group activities in online spaces.SOLUTION: A server sets at least one of multiple client terminals as a host terminal and sets the other client terminals as guest terminals, acquires the viewpoint mode of each guest terminal, and, in response to receiving viewpoint movement information from any of the client terminals, generates video corresponding to the viewpoint after the movement and transmits it to the target client terminal. Here, the viewpoint mode includes a synchronized viewpoint mode that displays video from the same viewpoint as the host terminal and an independent viewpoint mode that displays video from an independent viewpoint. When the server receives information about viewpoint movement from the host terminal, it transmits the video corresponding to the viewpoint after the movement as host viewpoint video to the host terminal and to the guest terminal operating in synchronized viewpoint mode.SELECTED DRAWING: Figure 7

Description

[Technical Field]

[0001] The present invention relates to a technique for distributing viewpoint-switchable content video over a network. [Background technology]

[0002] There is known a technology in which multiple participants use their own devices to join an online space created on a computer and share video, audio, and experiences in real time. Such technology is used in virtual reality (VR), augmented reality (AR), mixed reality (MR), cross reality (XR), metaverse, etc. For example, Patent Document 1 proposes an online tour system using VR. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Publication No. 2023-43568 Summary of the Invention [Problem to be solved by the invention]

[0004] While the technology for sharing online spaces and the user interfaces for it have evolved significantly since the past, they still cannot replicate the real-world experience, and much room for improvement remains. Consider, for example, a group activity such as sightseeing, walking, or sightseeing. In the real world, it is common for the entire group to move together and see what each person likes, or to look at something that catches someone's attention or what a guide is explaining, or to step away from the group and look at something else. Meanwhile, in online spaces, it is possible to individually show each participant a video from their preferred viewpoint (called "independent viewpoints") or to show all participants the same viewpoint (called "synchronized viewpoints"). However, no system has yet realized a hybrid mechanism of independent and synchronized viewpoints that is highly user-friendly or approaches a real-world experience.

[0005] For example, the system in Patent Document 1 also has a function that allows customers on an online tour to locally change the field of view on their own device (corresponding to an independent viewpoint) and a function that allows staff to change the field of view on the staff device and have the change reflected on all customer devices (corresponding to a synchronized viewpoint). However, if changes in the field of view on the staff device are forcibly and mechanically reflected on customer devices, as in the system in Patent Document 1, the image displayed on the customer device may change suddenly, causing viewers to feel uncomfortable or disturbed. Furthermore, if the image continues to change automatically while they are looking at something that interests them, it can lead to a decrease in customer satisfaction and result in a sense of difference from the real experience.

[0006] The present invention has been made in consideration of the above-mentioned circumstances, and has as its object to provide a new technology suitable for group activities in online spaces. [Means for solving the problem]

[0007] In order to achieve the above object, an online distribution system according to the present invention is an online distribution system for distributing video of viewpoint-switchable content over a network, and includes a server capable of communicating with a plurality of client terminals over the network, and the server sets at least one of the plurality of client terminals as a host terminal and sets the other client terminals as guest terminals, acquires the viewpoint mode of each guest terminal, and transmits viewpoint movement information from any of the plurality of client terminals. and in response to receiving the information on the movement of the viewpoint, generate an image corresponding to the viewpoint after the movement and transmit it to a target client terminal. Here, the viewpoint mode includes a synchronized viewpoint mode in which an image from the same viewpoint as that of the host terminal is displayed, and an independent viewpoint mode in which an image from an independent viewpoint is displayed regardless of the viewpoint of the host terminal, and when the server receives information on the movement of the viewpoint from the host terminal, it is preferable that the server transmits the image corresponding to the viewpoint after the movement as a host viewpoint image to the host terminal and to a guest terminal operating in the synchronized viewpoint mode.

[0008] With this configuration, a guest terminal operating in independent viewpoint mode can view video from an independent viewpoint independent of the viewpoint of the host terminal, while a guest terminal operating in synchronized viewpoint mode can view video from the same viewpoint as the host terminal in synchronization with the movement of the viewpoint on the host terminal. By allowing each guest terminal to set its own viewpoint mode and differently controlling the viewpoint on each guest terminal depending on the viewpoint mode, a viewing experience can be provided that meets each user's wishes and preferences (e.g., whether they want to watch their own favorite content or the same content as the host). This enables group activities in an online space that are closer to real-life experiences.

[0009] When the server receives information about the movement of the viewpoint from a guest terminal operating in the independent viewpoint mode, it may transmit an image corresponding to the viewpoint after the movement as a guest viewpoint image to the guest terminal that is the sender of the information about the movement of the viewpoint.

[0010] The server may be able to change the client terminal set as the host terminal in response to a request from the client terminal or spontaneously. By making it possible to change the host role, group behavior closer to a real experience can be realized.

[0011] The guest screen displayed on the guest terminal may include a video display area for displaying video and a mode switching UI for allowing the user to switch between synchronized viewpoint mode and independent viewpoint mode. When synchronized viewpoint mode is selected, the host viewpoint video received from the server may be displayed in the video display area of ​​the guest screen, and when independent viewpoint mode is selected, the guest viewpoint video received from the server may be displayed in the video display area of ​​the guest screen. By using such a guest screen, the user can freely switch between independent viewpoint and synchronized viewpoint according to their own interests and preferences.

[0012] The guest screen may display a sign in or around the video display area indicating whether the video currently displayed in the video display area is the host's perspective video or the guest's perspective video. Such a sign allows the guest to recognize at a glance whether the currently displayed video is the host's perspective video or the guest's perspective video. As long as the guest recognizes that they are viewing the guest's perspective video, they will not be confused even if what the host is saying does not match the content of the displayed video.

[0013] The host screen displayed on the host terminal may include a host video display area in which the host viewpoint video is displayed, and a simultaneous synchronization UI that allows an operation to switch the viewpoint mode of all guest terminals to a synchronized viewpoint mode.

[0014] The host screen displayed on the host terminal may include a host video display area where the host's viewpoint video is displayed and a guest video display area where the video displayed on each guest terminal is displayed. This host screen allows the host to understand what the other guests are looking at (what they are interested in), and can use this information to plan the guide route and topics.

[0015] The host screen may display a sign in or around the guest video display area indicating whether the video displayed on each guest terminal is the host's perspective video or the guest's perspective video. Such a sign allows the host to easily understand the status of the other guests (for example, whether everyone is interested in what the host is saying or not).

[0016] When the guest video display area on the host screen where the guest viewpoint video is displayed is selected by the user, information for matching the viewpoint of the host terminal with the viewpoint of the guest viewpoint video may be transmitted to the server. By using such a function, it becomes possible to instantly shift the host's viewpoint to an object that is attracting the guest's interest.

[0017] An online distribution method according to the present invention is an online distribution method for distributing video of viewpoint-switchable content over a network, the method comprising: a server capable of communicating with a plurality of client terminals over the network; setting at least one of the plurality of client terminals as a host terminal and setting the other client terminals as guest terminals; acquiring a viewpoint mode for each guest terminal; and, in response to receiving information of a viewpoint change from one of the plurality of client terminals, generating video corresponding to the viewpoint after the change and transmitting the video to the target client terminal. Here, the viewpoint modes include a synchronized viewpoint mode in which video from the same viewpoint as the host terminal is displayed, and an independent viewpoint mode in which video from a viewpoint independent of the viewpoint of the host terminal is displayed. When the server receives the information of the viewpoint change from the host terminal, it may transmit the video corresponding to the viewpoint after the change as host viewpoint video to the host terminal and to guest terminals operating in the synchronized viewpoint mode.

[0018] The present invention can be understood as an online distribution system, online space system, online event system, online tour system, online experience system, XR system, metaverse system, or a server or terminal constituting part of these systems, having at least a part of the above-mentioned configuration or the configuration described below. The present invention may also be understood as an online distribution method, online space providing method, online event providing method, online tour providing method, online experience providing method, XR experience providing method, metaverse providing method, or the like, having at least a part of the above-mentioned processing or the processing described below. The present invention may also be understood as a program that causes a computer to function as these systems, a program that causes a computer to execute these methods, or a computer-readable storage medium that non-temporarily stores such a program. [Effects of the Invention]

[0019] According to the present invention, a new technology suitable for group activities in an online space can be provided. [Brief explanation of the drawings]

[0020] [Figure 1] FIG. 1 shows an overview of the online distribution system. [Figure 2] Figure 2 is an example of the top screen of an online tour. [Figure 3] Figure 3 is an example of a guest screen displayed on a guest's device while participating in an online tour. [Figure 4] Figure 4 is an example of the host screen displayed on the host terminal while participating in an online tour. [Figure 5] FIG. 5 is a diagram showing a schematic diagram of the distribution control of the voice of the host and the voice of the guest in the online distribution system. [Figure 6] FIG. 6 is a flowchart of the mode switching process in the guest terminal. [Figure 7] FIG. 7 is a flowchart of the viewpoint video generation and transmission process in the server. [Figure 8] FIG. 8 is a diagram showing the flow of the video update process when the viewpoint is moved in the host terminal. [Figure 9] FIG. 9 is a diagram showing the flow of the video update process when the viewpoint is moved on a guest terminal operating in the independent viewpoint mode. DETAILED DESCRIPTION OF THE INVENTION

[0021] The present invention relates to a system for distributing video of viewpoint-switchable content over a network, and is preferably applicable to online events in which multiple users engage in group activities in an online space, such as online tours, online observation tours, online classes, online seminars, metaverses, etc. In the following embodiments, a specific application example of the present invention will be described using the example of an online tour.

[0022] <System configuration> FIG. 1 is a diagram showing an overall view of an online distribution system 1. As shown in FIG.

[0023] The online distribution system 1 includes a server 10 that provides content related to online tours, and multiple client terminals 11 used by multiple users participating in the online tour. The server 10 and the client terminal 11 can send and receive data to and from each other via the Internet. As will be described later, host authority is granted to at least one of the multiple client terminals 11. In this specification, a client terminal 11 that has been granted host authority is referred to as a "host terminal," a user using the host terminal is referred to as a "host," a client terminal 11 that has not been granted host authority is referred to as a "guest terminal," and a user using the guest terminal is referred to as a "guest." Host authority is the authority to distribute video from one's own perspective and one's own audio to guest terminals, and is granted to, for example, a person in a role such as a tour guide or group leader (chaperone).

[0024] The server 10 is a device that provides tour participants with a simulated travel experience using video and audio. The server 10 can be configured, for example, as a general-purpose computer system equipped with a processor (CPU), memory, storage, communication devices, etc., and the functions of the server 10, which will be described later, are realized by the processor executing a program loaded from the storage into the memory. The server 10 may be configured as a single computer, or may be configured as a combination of multiple computers as in distributed computing, or may be dynamically allocated resources as in cloud computing.

[0025] The client terminal 11 is a terminal device used by tour participants. The client terminal 11 can be configured, for example, by a general-purpose computer equipped with a processor (CPU), memory, storage, a communication device, a display device, an input device, a microphone, a speaker, and the like. The client terminal 11 can be, for example, a personal computer, a tablet terminal, a smartphone, a game console, smart glasses, a VR terminal, an XR terminal, or the like. The functions of the client terminal 11 are realized by a pre-installed native app or a web app that can be used on a web browser. The native app can be downloaded from the server 10 or an app store.

[0026] <Online Tour> An overview of the online tour provided by the online distribution system 1 will be described with reference to Figs. 2 and 3. An online tour is a web service that uses video content, VR / XR content, etc. to allow participants to watch videos of tourist spots and hear the audio of a guide on their devices, providing a simulated travel experience without actually visiting the tourist spots. In this embodiment, content that allows the viewpoint to be arbitrarily switched is used. This type of content For example, it may be generated based on video data captured in the real world using a 360-degree camera, or it may be generated based on virtual three-dimensional data such as a metaverse space.

[0027] Figure 2 is an example of the top screen of an online tour. When a person wishing to participate in an online tour accesses the online tour site using a browser or a dedicated app from their client terminal 11, the top screen of Figure 2 is displayed. When "Login" is selected in the upper right corner of the top screen, a login screen (not shown) is displayed, and the user is authenticated using an ID and password to log in. New users can register by entering personal information (such as name and address), a nickname to be used on the tour, and a payment method (such as a credit card) on a designated registration screen (not shown).

[0028] The top screen in Figure 2 displays a list of multiple tours in tiles. The tiles representing each tour display the tour name, destination, tour date and time, thumbnail images of tourist attractions, and so on. By selecting a desired tour from this list, the user can participate in or make a reservation for that tour. In other words, if a tour that is currently being held is selected, the client terminal 11 will be connected to the online space for that tour, and video and audio of that tour will be transmitted to the client terminal 11. On the other hand, if a tour that has not yet started is selected, the user can check the details of that tour and make a reservation for that tour.

[0029] Host authority is set for at least one of the multiple client terminals 11 participating in the tour. Hereinafter, for the sake of distinction, a client terminal 11 that has been set with host authority will be referred to as a "host terminal 11h," and other client terminals 11 will be referred to as "guest terminals 11g."

[0030] 3 shows an example of a guest screen displayed on the guest terminal 11g. The guest screen is made up of a video display area 30, a sub-window 31, a main menu 32, a location menu 33, and a mode switching button .

[0031] Video display area 30 is an area where video received from server 10 is displayed. Sub-window 31 is an area where the video of the host's face and information about individual calls (described later) are displayed. The display position and size of sub-window 31 may be changed, and it may be possible to switch between display and non-display.

[0032] The main menu 32 includes an individual call button 321, a cart button 322, a user button 323, a help button 324, an exit button 325, and the like. The individual call button 321 is used to start an individual call with another guest (details will be described later). Pressing the cart button 322 transitions to a purchase screen for souvenirs sold on the XR content. The user button 323 is used to check or change user properties, etc. Pressing the help button 324 displays instructions on how to use the tour screen and Q&A, etc. Pressing the exit button 325 exits the tour.

[0033] The location menu 33 is a menu that displays a list of tourist spots and points of interest included in the XR content. When a guest selects an item from the location menu 33, the image in the video display area 30 switches to a viewpoint image of that item, or an explanation of the item is superimposed on the video display area 30.

[0034] The mode switching button 34 is a UI (user interface) for allowing the user to switch between viewpoint modes. In this embodiment, the mode switching button 34 is a toggle button, and each time it is pressed, it switches between "synchronized viewpoint mode" and "independent viewpoint mode." The synchronized viewpoint mode is a mode in which an image from the same viewpoint as that of the host terminal 11h (hereinafter referred to as "host viewpoint image") is displayed in the image display area 30 of the guest terminal 11g, and the independent viewpoint mode is a mode in which an image from the same viewpoint as that of the host terminal 11h (hereinafter referred to as "host viewpoint image") is displayed in the image display area 30 of the guest terminal 11g. In this mode, video from an independent viewpoint (hereinafter referred to as "guest viewpoint video") is displayed in the video display area 30 of the guest terminal 11g, regardless of the viewpoint of the host terminal 11h. The host operates the viewpoint (pan / tilt / zoom) of the video on the host terminal 11h, while providing explanations that are tailored to the displayed video. When the guest switches to "synchronized viewpoint" using the mode switching button 34 on the guest terminal 11g, the same host viewpoint video as that of the host terminal 11h is displayed in the video display area 30, allowing the guest to listen to the host's explanatory voice while viewing the same video. On the other hand, when the guest switches to "independent viewpoint" using the mode switching button 34, the guest can use an input device such as a mouse or touch panel to zoom in and out of the video displayed in the video display area 30 or change the viewpoint to view the guest viewpoint video. Furthermore, if the guest is wearing a head-mounted display, the viewpoint of the video is controlled to switch in conjunction with the guest's head and line of sight movements.

[0035] On the guest screen of this embodiment, when the synchronized viewpoint mode is selected, the outer frame of the video display area 30 is displayed in red, and the label of the mode switching button 34 displays "synchronized viewpoint." On the other hand, when the independent viewpoint mode is selected, the outer frame of the video display area 30 is displayed in green (for illustration purposes, FIG. 3 shows it in dashed lines instead of green), and the label of the mode switching button 34 displays "independent viewpoint." Here, the color of the outer frame of the video display area 30 and the label of the mode switching button 34 serve as indicators that visually indicate whether the video displayed in the video display area 30 is a host-viewpoint video or a guest-viewpoint video. Such indicators make it possible to recognize at a glance whether the currently displayed video is a host-viewpoint video or a guest-viewpoint video. Note that the indicators are not limited to these examples and may be, for example, icons, text, annotations, or the like.

[0036] 4 is an example of a host screen displayed on the host terminal 11h. The host screen is composed of a host video display area 40, a sub-window 41, a main menu 42, a location menu 43, a simultaneous synchronization button 44, and guest video display areas 45a to 45e.

[0037] The host video display area 40 is an area where the host's own viewpoint video received from the server 10 is displayed. The sub-window 41, main menu 42, and location menu 43 are the same as the sub-window 31, main menu 32, and location menu 33 on the guest screen (FIG. 3). The host can manipulate the viewpoint of the host viewpoint video displayed in the host video display area 40.

[0038] The simultaneous synchronization button 44 is a UI for switching the viewpoint mode of all guest terminals 11g to synchronized viewpoint mode. When the host presses the simultaneous synchronization button 44 on the host terminal 11h, all guest terminals 11g are set to synchronized viewpoint mode, and the host viewpoint video is distributed to all guest terminals 11g. For example, the simultaneous synchronization button 44 can be conveniently used when the host wants to direct everyone's attention to the same object.

[0039] The guest video display areas 45a to 45e are areas where the video displayed on each guest terminal 11g is displayed. The guest video display areas 45a, 45b, and 45d corresponding to the guest terminal 11g operating in the synchronized viewpoint mode display the same host viewpoint video as the host terminal 11h, and the guest video display areas 45c and 45e corresponding to the guest terminal 11g operating in the independent viewpoint mode display the guest viewpoint video of each guest terminal 11g. On the host screen of this embodiment, the outer frames of the guest video display areas 45a, 45b, and 45d where the host viewpoint video is displayed are displayed in red, and the outer frames of the guest video display areas 45c and 45e where the guest viewpoint video is displayed are displayed in green. Here, the color of the outer frame of the guest video display area serves as an indicator that visually indicates whether the video displayed on the corresponding guest terminal 11g is the host viewpoint video or the guest viewpoint video. Such an indicator allows the host to easily grasp the status of the other guests (for example, whether everyone is interested in what the host is saying or not). The indicators are The present invention is not limited to these examples, and may be, for example, an icon, text, or annotation.

[0040] <Individual call> The individual call function of the online distribution system 1 will be described with reference to FIG. 5. The individual call function is a function that allows some of the guests participating in an online tour to individually engage in two-way audio communication while viewing the tour video and the host's audio. The audio of the individual call is not distributed to the other guests. This function is used, for example, for private conversations between friends who participated in a tour together.

[0041] Fig. 5 shows a schematic diagram of the distribution control of the voices of the host and the guests in the online distribution system 1. Fig. 5 shows an example in which a host 50 and three guests 51 to 53 are participating in an online tour. For ease of explanation, only four participants are shown, but in actual operation, the number of participants can range from several to several tens of people, or even more than a hundred people.

[0042] As shown in FIG. 5A, the online tour can be thought of as a single room R1 (conference room) in which the host 50 and all of the guests 51 to 53 are participants. Here, the room R1 consisting of all of the participants in the online tour is referred to as the "overall room." In the overall room R1, the server 10 receives the voice of the host 50 from the host terminal 11h and controls the transmission of that voice uniformly to the guest terminals 11g of all of the guests 51 to 53 participating in the overall room R1. This allows the guests 51 to 53 participating in the online tour to hear the voice of the host 50. Note that, to reduce traffic, the server 10 does not need to receive the voice of the guest terminal 11g in the overall room R1.

[0043] 5B shows distribution control when an individual call is made between guest 51 and guest 52. The individual call between guest 51 and guest 52 can be realized by setting up an individual room R2 in which only guest 51 and guest 52 participate within the general room R1. In the individual room R2, the server 10 transmits the audio received from the guest terminal 11g of guest 51 to the guest terminal 11g of the other guest 52, and conversely transmits the audio received from the guest terminal 11g of guest 52 to the guest terminal 11g of guest 51. At this time, the server 10 does not transmit the audio received from the participants of the individual room R2 (guests 51 and 52) to the guest terminal 11g of guest 53 who is not participating in the individual room R2. This distribution control enables two-way audio communication only between guest 51 and guest 52 who participate in the individual room R2.

[0044] The online distribution system 1 can be constructed using, for example, OME (OvenMediaEngine) based on WebRTC SFU (Web Real-Time Communication Selective Forwarding Unit). Also, individual calls between guest terminals can be constructed using open source software such as LiveKit.

[0045] <Viewpoint mode switching process> FIG. 6 is a flowchart showing an example of a viewpoint mode switching process in the guest terminal 11g.

[0046] When the processor of the guest terminal 11g detects that the mode switching button 34 on the guest screen has been pressed by the user (step S60), the processor branches the process depending on the viewpoint mode after switching (step S61).

[0047] When the mode is switched to the independent viewpoint mode, the processor of the guest terminal 11g notifies the server 10 of the change to the independent viewpoint mode together with information for identifying the guest terminal 11g (step S62). Any information that can uniquely identify the guest terminal 11g may be used, such as the ID of the user participating in the online tour using the guest terminal 11g, the IP address of the guest terminal 11g, or the session ID between the guest terminal 11g and the server 10. Although not shown, the server 10 stores information that identifies the guest terminal 11g in association with information about the viewpoint mode of the guest terminal 11g. The processor of the guest terminal 11g then rewrites the label of the mode switching button 34 to "independent viewpoint," draws the outer frame of the video display area 30 on the guest screen in green (step S63), and ends the process.

[0048] On the other hand, if the mode is switched to the synchronous viewpoint mode, the processor of the guest terminal 11g notifies the server 10 of the change to the synchronous viewpoint mode, along with information identifying the guest terminal 11g (step S64). In response, the server 10 transmits the host viewpoint video. When the processor of the guest terminal 11g receives the host viewpoint video from the server 10 (step S65), it displays the host viewpoint video in the video display area 30 of the guest screen (step S66). Then, the processor of the guest terminal 11g rewrites the label of the mode switching button 34 to "synchronous viewpoint," draws the outer frame of the video display area 30 of the guest screen in red (step S67), and ends the processing. As a result, the video displayed on the guest screen switches from the guest viewpoint video to the host viewpoint video.

[0049] <Perspective video generation and transmission processing> FIG. 7 is a flowchart showing an example of a viewpoint video generation and transmission process in the server 10.

[0050] When the processor of the server 10 receives information about a movement of the viewpoint from any of the client terminals 11 (step S70), it checks whether the client terminal 11 that transmitted the information has host authority, i.e., whether it is the host terminal 11h or the guest terminal 11g (step S71). If information identifying the client terminal 11 that has host authority is managed in a database in the server 10, the processor of the server 10 may determine whether the information transmission source is the host terminal 11h or the guest terminal 11g by referring to the database. Alternatively, if information for determining whether the client terminal 11 has host authority is sent from the client terminal 11 along with the information about the movement of the viewpoint, the processor of the server 10 may determine whether the information transmission source is the host terminal 11h or the guest terminal 11g based on the information.

[0051] When receiving information about the movement of the viewpoint from the host terminal 11h, the processor of the server 10 generates an image corresponding to the viewpoint after the movement (step S72). Note that various technologies have been proposed for generating an image corresponding to a desired viewpoint from VR content, XR content, or the like, and any of these technologies may be used in the present system 1. The processor of the server 10 returns the image generated in step S72 to the host terminal 11h that sent the information as a "host viewpoint image" (step S73). Furthermore, the processor of the server 10 also transmits the host viewpoint image generated in step S72 to the guest terminal 11g that is operating in the synchronized viewpoint mode among the multiple guest terminals 11g (step S74). As a result, an image of the viewpoint specified by the user of the host terminal 11h is distributed to the host terminal 11h and the guest terminal 11g that is operating in the synchronized viewpoint mode, and the synchronized images are displayed on each terminal.

[0052] When information on the movement of the viewpoint is received from the guest terminal 11g, the processor of the server 10 checks whether the viewpoint mode of the guest terminal 11g that sent the information is the independent viewpoint mode or the synchronous viewpoint mode (step S75). If the information on the viewpoint mode of each guest terminal 11g is managed in a database in the server 10, the processor of the server 10 may determine the viewpoint mode of the guest terminal 11g that sent the information by referring to the database. Alternatively, the processor of the server 10 may determine the viewpoint mode together with the information on the movement of the viewpoint from the client terminal 11. When information for viewing the view is sent, the processor of the server 10 may determine the viewpoint mode of the guest terminal 11g that is the source of the information based on the information.

[0053] If the information sender is the guest terminal 11g operating in the independent viewpoint mode, the processor of the server 10 generates video corresponding to the viewpoint after the movement (step S76). The same technique as in step S72 may be used to generate the viewpoint video. The processor of the server 10 returns the video generated in step S76 to the guest terminal 11g that sent the information as a "guest viewpoint video" (step S77). The processor of the server 10 also sends the guest viewpoint video generated in step S76 to the host terminal 11h together with information identifying the guest terminal 11g that sent the information (step S78). As a result, video of the viewpoint specified by the user of the guest terminal 11g is distributed to the guest terminal 11g and displayed on the guest screen. The same video is then displayed in the guest video display area of ​​the host screen of the host terminal 11h. As shown in FIG. 4, since the guest video display areas 45a to 45e are smaller than the host video display area 40, the processor of the server 10 may generate thumbnail images by reducing the guest viewpoint video and transmit the thumbnail images to the host terminal 11h.

[0054] If the information transmission source is the guest terminal 11g operating in the synchronous viewpoint mode, the processor of the server 10 does not generate or transmit viewpoint video, and ends the process.

[0055] <Example of video update processing> An example of the flow of the video update process in each client terminal is shown in Figures 8 and 9. Figure 8 shows the flow of the video update process when the viewpoint is moved in the host terminal 11h, and Figure 9 shows the flow of the video update process when the viewpoint is moved in the guest terminal 11g operating in the independent viewpoint mode.

[0056] 8, when a viewpoint movement operation is performed on the host screen of the host terminal 11h, information on the movement of the viewpoint is transmitted to the server 10 (step S80). An example of a viewpoint movement operation is given below.

[0057] Clicking or tapping on a point in the video displayed in the host video display area 40 = Jumping the viewpoint to the clicked or tapped point Drag or touch move operation within the video displayed in the host video display area 40 = slide the viewpoint (pan / tilt) Rotating the mouse wheel upward or pinching out within the image displayed in the host image display area 40 = Zoom in Rotating the mouse wheel downward or pinching in on the video displayed in the host video display area 40 = Zoom out Clicking or tapping on any of the guest video display areas 45a to 45e = Jump to the same viewpoint as the guest video displayed in the clicked or tapped area

[0058] The server 10 generates a host viewpoint video based on the information on the movement of the viewpoint received from the host terminal 11h (step S81). This process corresponds to steps S70 to S72 in FIG.

[0059] The generated host viewpoint video is transmitted to the host terminal 11h and the guest terminal 11g operating in the synchronous viewpoint mode (step S82). This process corresponds to steps S73 to S74 in FIG.

[0060] In the host terminal 11h, the host viewpoint video received from the server 10 is displayed as a host image on the host screen. The host viewpoint video received from the server 10 is displayed in the video display area 40 (step S83). In the guest terminal 11g operating in the synchronized viewpoint mode, the host viewpoint video received from the server 10 is displayed in the video display area 30 of the guest screen (step S84). As a result, synchronized videos are displayed on the host terminal 11h and the guest terminal 11g operating in the synchronized viewpoint mode.

[0061] 9, when a viewpoint shift operation is performed on the guest screen of the guest terminal 11g operating in the independent viewpoint mode, information about the viewpoint shift is transmitted to the server 10 (step S90). An example of a viewpoint shift operation is given below.

[0062] Clicking or tapping on a point in the image displayed in the image display area 30 = Jumping the viewpoint to the clicked or tapped point Drag or touch move within the image displayed in the image display area 30 = Slide the viewpoint (pan / tilt) Rotating the mouse wheel upward or pinching out within the image displayed in the image display area 30 = Zoom in Rotating the mouse wheel downwards or pinching in on the image displayed in the image display area 30 = Zoom out

[0063] The server 10 generates a guest viewpoint video based on the information on the movement of the viewpoint received from the guest terminal 11g (step S91). This process corresponds to steps S70, S71, S75, and S76 in FIG.

[0064] The generated guest-viewpoint video is transmitted to guest terminal 11g, the information transmission source (step S92). This process corresponds to step S77 in FIG. 7. In guest terminal 11g, the guest-viewpoint video received from server 10 is displayed in video display area 30 of the guest screen (step S93). The guest-viewpoint video is also transmitted to host terminal 11h together with information identifying guest terminal 11g, the information transmission source (step S94). This process corresponds to step S78 in FIG. 7. In host terminal 11h, the guest-viewpoint video received from server 10 is displayed in one of guest video display areas 45a to 45e of the host screen (the guest video display area corresponding to guest terminal 11g, the information transmission source) (step S95).

[0065] According to the online distribution system 1 of the present embodiment described above, a guest terminal 11g operating in independent viewpoint mode can view video from a viewpoint independent of the viewpoint of the host terminal 11h, while a guest terminal 11g operating in synchronized viewpoint mode can view video from the same viewpoint as the host terminal 11h in synchronization with the movement of the viewpoint of the host terminal 11h. By allowing a viewpoint mode to be set for each guest terminal 11g and differently controlling the viewpoint of each guest terminal 11g depending on the viewpoint mode, a viewing experience can be provided that meets each user's wishes and preferences (e.g., whether they want to watch their own favorite content or the same content as the host). This enables group activities in an online space that are closer to real-life experiences.

[0066] Furthermore, by providing a mode switching UI on the guest screen displayed on the guest terminal 11g, users can freely switch between independent viewpoint mode and synchronized viewpoint mode according to their own interests and preferences. For example, in a group tour, users can listen to the voice of the host who is acting as a guide, but move around while looking wherever they like in independent viewpoint mode, and only when they become interested in what the host is saying can they switch to synchronized viewpoint mode and look at what the host is explaining. This kind of operation can be easily realized.

[0067] <Other> The above embodiment merely describes one application example of the present invention, and the scope of the present invention does not include the above embodiment. For example, although the above embodiment describes an example in which the present invention is applied to an online tour, the present invention can also be applied to other online events.

[0068] Furthermore, in the above embodiment, the client terminal 11 having host authority is fixed. However, the client terminal 11 for which host authority is set may be changed in response to a request from the client terminal 11 or voluntarily by the server 10. For example, a UI for acquiring or transferring host authority may be provided on the guest screen or the host screen. Alternatively, the client terminal 11 having host authority may be automatically switched according to a predetermined tour scenario. By making it possible to change the host terminal, for example, in one scene, user A may act as the host and control the viewpoint and provide guidance, and in another scene, the host may be changed to user B, who may control the viewpoint and provide guidance. In this way, group activities can be carried out while freely changing the host role (i.e., guide or speaker) within a group. [Explanation of symbols]

[0069] 1: Online distribution system 10: Server 11: Client terminal 11h: Host terminal 11g: Guest terminal

Claims

1. An online distribution system for distributing viewpoint-switchable content video over a network, comprising: The system has a server that can communicate with multiple client terminals via a network, The server setting at least one of the plurality of client terminals as a host terminal and setting the other client terminals as guest terminals; Obtaining the viewpoint mode of each guest terminal; In response to receiving information on a movement of a viewpoint from any one of the plurality of client terminals, generating an image corresponding to the viewpoint after the movement and transmitting the image to a target client terminal; the viewpoint modes include a synchronized viewpoint mode in which an image from the same viewpoint as that of the host terminal is displayed, and an independent viewpoint mode in which an image from an independent viewpoint is displayed regardless of the viewpoint of the host terminal; When the server receives information about the movement of the viewpoint from the host terminal, the server transmits an image corresponding to the viewpoint after the movement as a host viewpoint image to the host terminal and the guest terminal operating in the synchronous viewpoint mode. Online distribution system.

2. When the server receives information about the movement of the viewpoint from the guest terminal operating in the independent viewpoint mode, the server transmits a video corresponding to the viewpoint after the movement as a guest viewpoint video to the guest terminal that is the sender of the information about the movement of the viewpoint. The online distribution system according to claim 1 .

3. The server is capable of changing the client terminal set as the host terminal in response to a request from the client terminal or spontaneously. The online distribution system according to claim 2 .

4. the guest screen displayed on the guest terminal includes a video display area in which a video is displayed, and a mode switching UI that allows a user to switch between a synchronized viewpoint mode and an independent viewpoint mode; When a synchronized viewpoint mode is selected, the host viewpoint video received from the server is displayed in the video display area of ​​the guest screen, When an independent viewpoint mode is selected, the guest viewpoint video received from the server is displayed in the video display area of ​​the guest screen. The online distribution system according to claim 2 .

5. the guest screen displays an indicator in or around the video display area indicating whether the video currently displayed in the video display area is the host-perspective video or the guest-perspective video; The online distribution system according to claim 4.

6. The host screen displayed on the host terminal includes a host video display area in which a host viewpoint video is displayed, and a simultaneous synchronization UI that enables an operation to switch the viewpoint mode of all guest terminals to a synchronized viewpoint mode. The online distribution system according to claim 2 .

7. The host screen displayed on the host terminal includes a host video display area where a host viewpoint video is displayed, and a guest video display area where the video displayed on each guest terminal is displayed. nothing, The online distribution system according to claim 2 .

8. The host screen displays an indicator in or around the guest video image display area, indicating whether the video image displayed on each guest terminal is the host viewpoint video image or the guest viewpoint video image. The online distribution system according to claim 7.

9. When the guest video display area in which the guest viewpoint video is displayed is selected by a user on the host screen, information for matching the viewpoint of the host terminal with the viewpoint of the guest viewpoint video is transmitted to the server. The online distribution system according to claim 7.

10. An online distribution method for distributing viewpoint-switchable content video over a network, comprising: A server that can communicate with multiple client terminals via a network setting at least one of the plurality of client terminals as a host terminal and setting the other client terminals as guest terminals; Obtaining the viewpoint mode of each guest terminal; In response to receiving information on a movement of a viewpoint from any one of the plurality of client terminals, generating an image corresponding to the viewpoint after the movement and transmitting the image to a target client terminal; the viewpoint modes include a synchronized viewpoint mode in which an image from the same viewpoint as that of the host terminal is displayed, and an independent viewpoint mode in which an image from an independent viewpoint is displayed regardless of the viewpoint of the host terminal; When the server receives information about the movement of the viewpoint from the host terminal, the server transmits an image corresponding to the viewpoint after the movement as a host viewpoint image to the host terminal and the guest terminal operating in the synchronous viewpoint mode. Online delivery methods.

Images