Computer program, method and server device

Abstract

To provide a system to distribute and / or display a moving image by using a tracking device with attachment or the like to a distributor's body.SOLUTION: A computer program can "force at least one processor to function in the following manner: with the computer program executed by at least the one processor loaded on a certain terminal, it runs on the basis of an own user who is the user of the certain terminal to display a first moving image expressed as seen from the own user's visual point, display an input object which functions as a user interface enabling input by the own user in combination with the first moving image, and change the displayed input object according to the type of the displayed first moving image."SELECTED DRAWING: Figure 9

Description

[Technical field]

[0001] The technology disclosed in this application relates to a computer program, a method, a server device, etc., used for distributing and / or displaying moving images. [Background technology]

[0002] Currently, a service called "REALITY" (Non-Patent Document 1) is known, which distributes videos including animations of avatars that move according to the movements of a distributor from a terminal device of the distributor to a terminal device of a viewer. [Prior art documents] [Patent documents]

[0003] [Non-Patent Document 1] “REALITY”, [online], February 25, 2022, REALITY Co., Ltd., [Retrieved February 28, 2022], Internet (https: / / reality.app / ) Summary of the Invention [Problem to be solved by the invention]

[0004] In recent years, it is expected that broadcasters will use not only smartphones but also various tracking devices, including head-mounted displays (HMDs) and AR glasses, which are worn on the broadcaster's body, as terminal devices.

[0005] Therefore, the technology disclosed in this application provides a method for distributing and / or displaying video using a tracking device that is attached to the body of the distributor. [Means for solving the problem]

[0006] A computer program according to one embodiment can be executed by at least one processor installed in a terminal device to cause the at least one processor to function as follows: when a first terminal device receives first movement data generated using a first tracking method based on the movement of a first broadcaster, generate a video including an animation of a first avatar of the first broadcaster based on the first movement data; when a second terminal device receives second movement data generated using a second tracking method different from the first tracking method based on the movement of a second broadcaster, generate a video including an animation of a second avatar of the second broadcaster based on the second movement data; when a second terminal device receives both the first movement data and the second movement data, generate a video including animation of the first avatar based on the first movement data and animation of the second avatar based on the second movement data, and display the video on a display unit.

[0007] A method according to one embodiment can be described as "a method executed by at least one processor mounted on a terminal device that executes computer-readable instructions, wherein the at least one processor, by executing the instructions, when a first terminal device receives first movement data generated using a first tracking method based on the movement of a first broadcaster, generates a video including an animation of a first avatar of the first broadcaster based on the first movement data; when a second terminal device receives second movement data generated using a second tracking method different from the first tracking method based on the movement of a second broadcaster, generates a video including an animation of a second avatar of the second broadcaster based on the second movement data; when a second terminal device receives both the first movement data and the second movement data, generates a video including an animation of the first avatar based on the first movement data and an animation of the second avatar based on the second movement data; and displaying the video on a display unit."

[0008] A computer program according to another embodiment can cause the at least one processor to function as follows: "When executed by at least one processor installed in a terminal device, the at least one processor can cause the at least one processor to function as follows: transmit first movement data generated by the terminal device using a first tracking method based on the movement of the user himself / herself to a server device; receive second movement data generated by a second terminal device using a second tracking method different from the first tracking method based on the movement of a second broadcaster via the server device; generate a video including an animation of a first avatar of the user himself / herself based on the first movement data and an animation of a second avatar of the second broadcaster based on the second movement data; and display the video on a display unit."

[0009] A method according to another aspect can be "a method executed by at least one processor mounted on a terminal device that executes computer-readable instructions, the at least one processor executing the instructions to send first movement data generated by the terminal device using a first tracking method based on the movement of the user to a server device, receiving second movement data generated by a second terminal device using a second tracking method different from the first tracking method based on the movement of a second broadcaster via the server device, generating a video including an animation of a first avatar of the user based on the first movement data and an animation of a second avatar of the second broadcaster based on the second movement data, and displaying the video on a display unit."

[0010] A server device according to one embodiment includes at least one processor, and when the at least one processor receives from a first terminal device first motion data generated by the first terminal device using a first tracking method based on a motion of a first distributor, the server device transmits the first motion data to a certain terminal device to cause the certain terminal device to generate a video including an animation of a first avatar of the first distributor based on the first motion data, and transmits second motion data generated by a second terminal device using a second tracking method different from the first tracking method based on a motion of a second distributor to the certain terminal device. the second action data is transmitted to the certain terminal device when the second action data is received from the certain terminal device, causing the certain terminal device to generate a video including an animation of the second avatar of the second distributor based on the second action data; and the first action data and the second action data are both received from the certain terminal device, causing the certain terminal device to generate a video including an animation of the first avatar based on the first action data and an animation of the second avatar based on the second action data.

[0011] A method according to yet another aspect is a method executed by at least one processor mounted on a server device that executes computer-readable instructions, the method including the steps of: when first motion data generated by a first terminal device using a first tracking method based on a motion of a first broadcaster is received from the first terminal device, transmitting the first motion data to a certain terminal device, and causing the certain terminal device to generate a video including an animation of a first avatar of the first broadcaster based on the first motion data; and causing a second terminal device to generate a video including an animation of a first avatar of the first broadcaster based on the first motion data, the video including an animation of a first avatar of the first broadcaster based on the first motion data. when second motion data generated using a two-tracking method is received from the second terminal device, transmitting the second motion data to the certain terminal device to cause the certain terminal device to generate a video including an animation of a second avatar of the second distributor based on the second motion data; and when both the first motion data and the second motion data are received, transmitting both the first motion data and the second motion data to the certain terminal device to cause the certain terminal device to generate a video including animation of the first avatar based on the first motion data and animation of the second avatar based on the second motion data.

[0012] A server device according to another aspect can be a server device having at least one processor, the at least one processor being configured to transmit a web page including a computer program to a certain terminal device, the computer program being executed by the certain terminal device that has received the web page to cause the certain terminal device to function in the following manner: when a first terminal device receives first movement data generated using a first tracking method based on the movement of a first distributor, a video including an animation of a first avatar of the first distributor based on the first movement data; when a second terminal device receives second movement data generated using a second tracking method different from the first tracking method based on the movement of a second distributor, a video including an animation of a second avatar of the second distributor based on the second movement data; and when the certain terminal device receives both the first movement data and the second movement data, a video including an animation of the first avatar based on the first movement data and an animation of the second avatar based on the second movement data.

[0013] A computer program according to a further embodiment includes a program for displaying a first moving image, the first moving image being displayed from a viewpoint of a user of the terminal device, the first moving image being displayed from a viewpoint of the ... The at least one processor can be caused to function so as to "display an input object functioning as a user interface enabling input by the user in combination with the first moving image, and to vary the displayed input object depending on the type of the first moving image being displayed."

[0014] A method according to yet another aspect can include "a method executed by at least one processor mounted on a terminal device that executes computer-readable instructions, the at least one processor executing the instructions to operate based on the actions of a user who is a user of the terminal device and display a first video represented from the perspective of the user, displaying an input object that functions as a user interface allowing input by the user in combination with the first video, and changing the displayed input object depending on the type of the first video being displayed."

[0015] A system according to one embodiment may be "a system including a terminal device and a server device connected to the terminal device via a communication line, wherein the terminal device is configured to transmit to the server device motion data generated based on motion of a user of the terminal device, operate based on the motion data, display a first moving image represented from the user's perspective, display input objects functioning as a user interface enabling input by the user in combination with the first moving image, and change the displayed input objects depending on the type of the first moving image displayed, and the server device is configured to transmit the motion data received from the terminal device to at least one other terminal device."

[0016] A method according to yet another embodiment can be "a method executed by a system including a terminal device and a server device connected to the terminal device via a communication line, the method including: the terminal device transmitting to the server device motion data generated based on motion of a user of the terminal device; the terminal device operating based on the motion data and displaying a first moving image represented from the user's perspective; the terminal device displaying, in combination with the first moving image, an input object functioning as a user interface enabling input by the user; the terminal device changing the displayed input object depending on the type of the first moving image displayed; and the server device transmitting the motion data received from the terminal device to at least one other terminal device."

[0017] In yet another embodiment, the server device can be described as "a server device having at least one processor, the at least one processor being configured to transmit a web page including a computer program to a terminal device, the computer program being executed by the terminal device receiving the web page, causing the terminal device to function in the following manner: operate based on the actions of a user of the terminal device, display a first video represented from the user's perspective, display input objects functioning as a user interface enabling input by the user in combination with the first video, and change the displayed input objects depending on the type of the first video being displayed."

[0018] A computer program according to yet another embodiment can cause the at least one processor to function as follows: "When executed by at least one processor mounted on a terminal device, the at least one processor can acquire first movement data generated based on the movement of a user of the terminal device using multiple tracking devices worn by the user, who is a user of the terminal device, operate based on the first movement data, display a first video that represents a virtual space as viewed from the perspective of the user, operate based on the first movement data, display a second video that represents the virtual space including the user's avatar as viewed from an isolated perspective at a distance from the user's avatar, and attach an object to the user's avatar or detach an object from the user's avatar in accordance with operation data generated based on the operation of the user."

[0019] A method according to yet another aspect can include "a method executed by at least one processor mounted on a terminal device that executes computer-readable instructions, the at least one processor executing the instructions to acquire first movement data generated based on a movement of a user of the terminal device using a plurality of tracking devices attached to the user, the user being a user of the terminal device; displaying a first video that operates based on the first movement data and represents a virtual space as viewed from the viewpoint of the user; displaying a second video that operates based on the first movement data and represents the virtual space including the user's avatar as viewed from an isolated viewpoint at a distance from the user's avatar; and attaching an object to the user's avatar or detaching an object from the user's avatar in accordance with operation data generated based on the operation of the user."

[0020] In yet another embodiment, a server device may be "a server device having at least one processor, the at least one processor being configured to: acquire motion data generated based on motion of a user of a terminal device using a plurality of tracking devices worn by the user of the terminal device, operate based on the motion data, generate a first video that represents a virtual space as viewed from the user's viewpoint, and transmit the first video to the terminal device; operate based on the motion data, generate a second video that represents the virtual space including the user's avatar as viewed from a distant viewpoint at a distance from the user's avatar, and transmit the second video to the terminal device; receive operation data generated based on the user's operation from the terminal device; and attach an object to the user's avatar or detach an object from the user's avatar based on the operation data."

[0021] A method according to yet another embodiment can include "a method executed by at least one processor mounted on a server device that executes computer-readable instructions, the at least one processor executing the instructions to obtain motion data generated based on motion of a user of a terminal device using a plurality of tracking devices worn by the user of the terminal device, generating and transmitting to the terminal device a first video that operates based on the motion data and depicts a virtual space as seen from the user's viewpoint, generating and transmitting to the terminal device a second video that operates based on the motion data and depicts the virtual space including the user's avatar as seen from a remote viewpoint at a distance from the user's avatar, receiving from the terminal device operation data generated based on the user's operation, and attaching an object to the user's avatar or detaching an object from the user's avatar based on the operation data."

[0022] A system according to another embodiment may be a system including "a terminal device and a server device connected to the terminal device via a communication line, wherein the terminal device is configured to: acquire motion data generated based on the motion of a user of the terminal device using a plurality of tracking devices worn by the user of the terminal device, operate based on the motion data, display a first video representing a virtual space as viewed from the user's viewpoint, operate based on the motion data, display a second video representing the virtual space including the user's avatar as viewed from a remote viewpoint at a distance from the user's avatar, and attach an object to the user's avatar or detach an object from the user's avatar in accordance with operation data generated based on the user's operation; and the server device is configured to transmit the motion data received from the terminal device to at least one other terminal device."

[0023] A method according to yet another embodiment can be "a method executed by a system including a terminal device and a server device connected to the terminal device via a communication line, the method including: the terminal device acquiring motion data generated based on motion of the user of the terminal device using a plurality of tracking devices worn by the user of the terminal device; the terminal device operating based on the motion data and displaying a first video depicting a virtual space as viewed from the user's viewpoint; the terminal device operating based on the motion data and displaying a second video depicting the virtual space including the user's avatar as viewed from a remote viewpoint at a distance from the user's avatar; attaching an object to the user's avatar or detaching an object from the user's avatar in accordance with operation data generated based on the user's operation; and the server device transmitting the motion data received from the terminal device to at least one other terminal device."

[0024] In yet another embodiment, the server device can be configured to "have at least one processor, the at least one processor being configured to transmit a web page including a computer program to a terminal device, the computer program being executed by the terminal device receiving the web page, to cause the terminal device to function as follows: acquire motion data generated based on the motion of the user using a plurality of tracking devices worn by the user of the terminal device, operate based on the motion data, display a first video representing a virtual space as viewed from the user's viewpoint, operate based on the motion data, display a second video representing the virtual space including the user's avatar as viewed from a remote viewpoint at a distance from the user's avatar, and attach an object to the user's avatar or detach an object from the user's avatar in accordance with operation data generated based on the user's operation." [Brief description of the drawings]

[0025] [Figure 1] FIG. 1 is a block diagram showing an example of a configuration of a communication system according to an embodiment. [Diagram 2] FIG. 2 is a block diagram showing an example of a hardware configuration of the terminal device 20 used in the communication system 1 shown in FIG. [Diagram 3] FIG. 3 is a schematic diagram showing an example of a device that can be connected to the terminal device 20 or the like in order to generate motion data based on the first tracking method in the communication system 1 shown in FIG. [Figure 4A] FIG. 4A is a schematic diagram illustrating a first case regarding provision of moving images using the platform realized by the communication system 1 shown in FIG. [Figure 4B] FIG. 4B is a schematic diagram illustrating a first case regarding provision of moving images using the platform realized by the communication system 1 shown in FIG. [Figure 4C] FIG. 4C is a schematic diagram illustrating a first case regarding provision of moving images using the platform realized by the communication system 1 shown in FIG. [Figure 4D] FIG. 4D is a schematic diagram illustrating a first case regarding provision of moving images using the platform realized by the communication system 1 shown in FIG. [Figure 5A] FIG. 5A is a schematic diagram illustrating a second case regarding provision of moving images using the platform realized by the communication system 1 shown in FIG. [Figure 5B] FIG. 5B is a schematic diagram illustrating a second case regarding provision of moving images using the platform realized by the communication system 1 shown in FIG. [Figure 5C] FIG. 5C is a schematic diagram illustrating a second case regarding provision of moving images using the platform realized by the communication system 1 shown in FIG. [Figure 5D] FIG. 5D is a schematic diagram illustrating a second case regarding provision of moving images using the platform realized by the communication system 1 shown in FIG. [Figure 6A] FIG. 6A is a schematic diagram illustrating a third case regarding provision of moving images using the platform realized by the communication system 1 shown in FIG. [Figure 6B] FIG. 6B is a schematic diagram illustrating a third case regarding provision of moving images using the platform realized by the communication system 1 shown in FIG. [Figure 6C] FIG. 6C is a schematic diagram illustrating a third case regarding provision of moving images using the platform realized by the communication system 1 shown in FIG. [Figure 6D] FIG. 6D is a schematic diagram illustrating a third case regarding provision of moving images using the platform realized by the communication system 1 shown in FIG. [Figure 7] FIG. 7 is a diagram showing an example of the first moving image displayed on a terminal device of the first distributor in the communication system 1 shown in FIG. [Figure 8] FIG. 8 is a diagram showing an example of the second moving image displayed on the terminal device of the first distributor in the communication system 1 shown in FIG. [Figure 9]FIG. 9 is a diagram showing another example of the first moving image and the second moving image displayed by the terminal device of the first distributor in the communication system 1 shown in FIG. [Figure 10] FIG. 10 is a diagram showing an example of the first moving image displayed by the terminal device of the viewer A in the communication system 1 shown in FIG. [Figure 11] FIG. 11 is a diagram showing another example of the first moving image displayed on the terminal device of the first distributor in the communication system 1 shown in FIG. [Figure 12] FIG. 12 is a diagram showing another example of the second moving image displayed on the terminal device of the first distributor in the communication system 1 shown in FIG. [Figure 13] FIG. 13 is a diagram showing an example of the first moving image displayed on a terminal device of a third distributor in the communication system 1 shown in FIG. [Figure 14] FIG. 14 is a flow diagram showing an example of an operation executed in the communication network 1 shown in FIG. [Figure 15] FIG. 15 is a diagram showing an example of a screen displayed when the terminal device in the communication system shown in FIG. 1 executes the "preparation mode." [Figure 16] FIG. 16 is a diagram showing an example of a screen displayed by a terminal device in the communication system shown in FIG. 1 when the terminal device executes the "viewing mode." [Figure 17] FIG. 17 is a diagram showing another example of a screen displayed when the terminal device in the communication system shown in FIG. 1 executes the "viewing mode." [Figure 18] FIG. 18 is a diagram showing an example of a screen displayed when the terminal device in the communication system shown in FIG. 1 executes the "joint distribution mode." [Figure 19] FIG. 19 is a diagram showing an example of a screen displayed when the terminal device in the communication system shown in FIG. 1 executes the "single distribution mode." [Figure 20] FIG. 20 is a diagram showing another example of a screen displayed when the terminal device in the communication system shown in FIG. 1 executes the "viewing mode." [Figure 21]FIG. 21 is a flow diagram illustrating another example of the operation executed by the communication system 1 shown in FIG. [Figure 22] FIG. 22 is a diagram showing an example of a screen displayed when the "closet mode" is executed by the terminal device in the communication system shown in FIG. [Figure 23] FIG. 23 is a diagram conceptually showing the viewpoint of a user (avatar) used when generating the first moving image in the communication network 1 shown in FIG. [Figure 24] FIG. 24 is a diagram conceptually showing an isolated viewpoint used when generating an isolated video in the communication network shown in FIG. [Diagram 25] FIG. 25 is a diagram showing another example of a screen displayed when the "closet mode" is executed by the terminal device in the communication system shown in FIG. [Figure 26] FIG. 26 is a diagram showing another example of a screen displayed when the "closet mode" is executed by the terminal device in the communication system shown in FIG. [Figure 27] FIG. 27 is a diagram showing another example of a screen displayed when the "closet mode" is executed by the terminal device in the communication system shown in FIG. [Figure 28] FIG. 28 is a diagram showing another example of a screen displayed when the terminal device in the communication system shown in FIG. 1 executes the "joint distribution mode." DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0026] This specification has been described in terms of various representative embodiments, which are not intended to be limiting in any way. As used in this application, the singular forms "a," "the," "said," "above," "the," "the," "this," and "it" can include the plural unless the plural is clearly not indicated. Additionally, the term "comprising" can mean "comprising" or "having." Additionally, the terms "coupled," "coupled," "connected," "connected," and "connected" encompass mechanical, electrical, magnetic, and optical ways of coupling, connecting, or joining objects together, among other ways, and do not exclude the presence of intermediate elements between objects so coupled, coupled, coupled, connected, connected, or connected.

[0027] The various systems, methods, and devices described herein should not be construed as being limited in any manner. Indeed, the present disclosure is directed to all novel features and aspects of each of the various disclosed embodiments, combinations of these various embodiments with each other, and combinations of portions of these various embodiments with each other. The various systems, methods, and devices described herein are not limited to a particular aspect, a particular feature, or a combination of such a particular aspect with a particular feature, nor do the products and methods described herein require that one or more particular advantages be present or problems be solved. Moreover, various features or aspects of the various embodiments described herein, or portions of such features or aspects, may be used in combination with each other.

[0028] Although operations of some of the various methods disclosed herein have been described in a particular order for convenience, it should be understood that description in such a manner encompasses rearranging the order of the operations, unless a particular order is required by specific text below. For example, operations described in a sequence may in some cases be rearranged or performed simultaneously. Furthermore, for purposes of simplicity, the accompanying drawings do not show the various ways in which the various items and methods described herein may be used with other items and methods. In addition, the present specification may use terms such as "generate," "generate," "display," "receive," "evaluate," and "deliver." These terms are high-level descriptions of the various actual operations that may be performed. The actual operations that correspond to these terms may vary depending on the particular implementation and may be readily discerned by one of ordinary skill in the art having the benefit of the disclosure herein.

[0029] Any theories of operation, scientific principles, or other theoretical descriptions presented herein in connection with the devices or methods of the present disclosure are provided for the purpose of better understanding and are not intended to limit the scope of the technology, and the devices and methods in the appended claims are not limited to devices and methods that operate in a manner described by such theories of operation.

[0030] Any of the various methods disclosed herein may be implemented and executed in a computer using computer-executable instructions stored on one or more computer-readable media (e.g., non-transitory computer-readable storage media, such as one or more optical media disks, volatile memory components, or non-volatile memory components), where the volatile memory components include, for example, DRAM or SRAM, and the non-volatile memory components include, for example, hard drives and solid-state drives (SSD). Additionally, the computer may include any computer available in the market, including, for example, smartphones and other mobile devices having hardware to perform computations.

[0031] Any such computer-executable instructions for implementing the techniques disclosed herein, along with any data generated and used during the implementation of the various embodiments disclosed herein, may be stored on one or more computer-readable media (e.g., non-transitory computer-readable storage media). Such computer-executable instructions may, for example, be part of a separate software application, or may be part of a software application accessed or downloaded via a web browser or other software application (such as a remote computing application). Such software may, for example, be executed on a single local computer (e.g., as a process executed on any suitable computer available on the market) or in a networked environment (e.g., the Internet, a wide area network, a local area network, a client-server network (such as a cloud computing network), or other such network) using one or more networked computers.

[0032] For clarity, only certain selected aspects of various software-based implementations are described. Other details that are well known in the art are omitted. For example, the techniques disclosed herein are not limited to a specific computer language or program. For example, the techniques disclosed herein may be implemented by software written in C, C++, Java, or any other suitable programming language. Similarly, the techniques disclosed herein are not limited to a specific computer or a specific type of hardware. Specific details of suitable computers and hardware are well known and need not be described in detail herein.

[0033] Moreover, any of the various such software-based embodiments (e.g., including computer-executable instructions for causing a computer to perform any of the various methods disclosed herein) may be uploaded, downloaded, or accessed in a remote manner by any suitable communication means, including, for example, the Internet, the World Wide Web, an intranet, a software application, cable (including fiber optic cable), magnetic communication, electromagnetic communication (including RF, microwave, and infrared communication), electronic communication, or other such communication means.

[0034] The term "gift" used in this application has the same concept as the term "token." Therefore, the technology described in this application can be understood by replacing the term "gift" with the term "token."

[0035] Various embodiments of the present invention will be described below with reference to the accompanying drawings. Note that common components in the drawings are given the same reference numerals. Also, please note that components shown in one drawing may be omitted in another drawing for convenience of explanation. Furthermore, please note that the attached drawings are not necessarily drawn to scale.

[0036] In this specification, the terms "video distribution" by a distributor or "distributing a video" by a distributor may include, for example, at least one of the aspects exemplified below. A terminal device of a distributor, either alone or in cooperation with terminal devices of other distributors, transmits data necessary for generating videos (e.g., motion data) to multiple viewers' terminal devices via a server device, etc. A terminal device of a certain distributor, either alone or in cooperation with terminal devices of other distributors, transmits generated videos to each viewer's terminal device via a server device, etc.

[0037] 1. Overview In brief, the communication system disclosed in this application may perform the following operations:

[0038] In a typical example in which a first distributor and a second distributor jointly distribute videos to each viewer, a terminal device of the first distributor can transmit first motion data generated using a first tracking method based on the motion of the first distributor to a server device, and a terminal device of the second distributor can transmit second motion data generated using a second tracking method (different from the first tracking method) based on the motion of the second distributor to the server device.

[0039] Here, the first tracking method may include at least a method of generating motion data indicating the coordinates and angles of a device worn on the streamer's head and at least one controller worn or held on the streamer's body other than the streamer's head. The device worn on the streamer's head may include, but is not limited to, an HMD and VR glasses. The at least one controller may include, but is not limited to, a hand controller held by the streamer's right hand, a hand controller held by the streamer's left hand, a controller worn on the streamer's waist, etc.

[0040] The second tracking method may include at least a method (such as a face tracking method) for generating motion data indicating the depth of each of multiple parts of the broadcaster's face and the coordinates and angle of the broadcaster's head.

[0041] The server device can transmit the first motion data and the second motion data to a plurality of terminal devices of viewers, a terminal device of the first distributor, and a terminal device of the second distributor.

[0042] Each of the multiple terminal devices, including the terminal device of each viewer, the terminal device of the first distributor, and the terminal device of the second distributor, can identify the first motion data and the second motion data received from the server device. Then, each of the multiple terminal devices can generate and display a video including an animation of the avatar of the first distributor based on the first motion data and an animation of the avatar of the second distributor based on the second motion data.

[0043] On the other hand, in an example in which a first distributor solely provides video to each viewer, the terminal device of the first distributor can transmit first movement data generated using a first tracking method based on the movement of the first distributor to the server device.

[0044] The server device can transmit the first motion data to terminal devices of a plurality of viewers and to a terminal device of the first distributor.

[0045] Each of the multiple terminal devices, including the terminal device of each viewer and the terminal device of the first distributor, can identify the first motion data received from the server device. Then, each of the multiple terminal devices can generate and display a video including an animation of an avatar of the first distributor based on the first motion data.

[0046] On the other hand, in an example in which a second distributor solely provides video to each viewer, the terminal device of the second distributor can transmit second movement data generated using a second tracking method based on the movement of the second distributor to the server device.

[0047] The server device can transmit the second motion data to terminal devices of a plurality of viewers and to a terminal device of the second distributor.

[0048] Each of the multiple terminal devices, including the terminal device of each viewer and the terminal device of the second distributor, can identify the second motion data received from the server device. Then, each of the multiple terminal devices can generate and display a video including an animation of an avatar of the second distributor based on the second motion data.

[0049] In each of the above-mentioned examples, a method (client rendering method) is adopted in which the generation of the moving image is executed by each of the plurality of terminal devices. However, a method (server rendering method) in which the generation of such a moving image is executed by the server device (or another server device that can be connected to the server device) and the moving image thus generated is transmitted to and displayed on the plurality of terminal devices may also be adopted. Alternatively, a method (browser rendering method) in which the generation of the moving image is executed by each of the plurality of terminal devices, which receives a web page from the server device (or another server device that can be connected to the server device) and executes a computer program included in the web page, may also be adopted.

[0050] 2. Overall configuration of the communication system Next, a configuration of a communication system that executes the above-mentioned operations will be described with reference to Fig. 1. Fig. 1 is a block diagram showing an example of the configuration of a communication system according to an embodiment.

[0051] As shown in Fig. 1, the communication system 1 can include at least one server device 30, a plurality of terminal devices 20 connectable to a communication network 10, and at least one server device 30 connectable to the communication network 10. Fig. 1 shows an example in which the plurality of terminal devices 20 include three terminal devices 20A, 20B, and 20C, but the plurality of terminal devices 20 can include any number (two or more) of terminal devices 20. Similarly, Fig. 1 shows an example in which the at least one server device 30 includes three server devices 30A, 30B, and 30C, but the at least one server device 30 can also include any number (one or more) of server devices 30.

[0052] The multiple terminal devices 20 may include at least one distributor's terminal device 20 and at least one viewer's terminal device 20. Each of the at least one distributor's terminal device 20 is capable of generating and transmitting motion data based on either the first tracking method or the second tracking method, and each of the at least one viewer's terminal device 20 is also capable of generating and transmitting motion data based on either the first tracking method or the second tracking method.

[0053] Furthermore, the multiple terminal devices 20 may be any information processing device, and may include, without being limited to, an HMD, VR glasses, a smartphone, a personal computer, a tablet, a personal digital assistant, a mobile phone, and combinations thereof.

[0054] The at least one server device 30 may be an information processing device capable of providing various services to each of the multiple terminal devices 20 that access the server device 30. The at least one server device 30 may be any information processing device, and may include, but is not limited to, a personal computer, a supercomputer, a workstation, a mainframe, and combinations thereof.

[0055] Furthermore, at least one server device 30 may include a web server device 30 that transmits a web page including a computer program to each of a plurality of terminal devices 20 that access this server device 30.

[0056] Furthermore, at least one server device 30 may be connectable to one another via the communication network 10.

[0057] The communication network 10 may include, without limitation, a cellular network, a wireless network, a landline network, the Internet, an intranet, a local area network (LAN), a wide area network (WAN), and / or an Ethernet network, where the wireless network may include, for example, RF connections via Bluetooth, WiFi (such as IEEE 802.11a / b / n), WiMax, cellular, satellite, laser, and / or infrared.

[0058] 3. Hardware configuration of terminal device 20 Next, the configuration of the terminal device 20 used in the above-mentioned communication system 1 will be described with reference to Fig. 2. Fig. 2 is a block diagram showing an example of the hardware configuration of the terminal device 20 used in the communication system 1 shown in Fig. 1 (note that in Fig. 2, the reference numerals in parentheses are described in relation to the server device 30 as described later).

[0059] 2, each terminal device 20 can mainly include a central processing unit 21, a main memory device 22, an input / output interface device 23, an input device 24, an auxiliary memory device 25, and an output device 26. These devices are connected to each other by a data bus and / or a control bus.

[0060] The central processing unit 21 is referred to as a "CPU" and can perform operations on instructions and data stored in the main memory 22 and store the results of the operations in the main memory 22. Furthermore, the central processing unit 21 can control an input device 24, an auxiliary memory device 25, an output device 26, etc. via an input / output interface device 23. The terminal device 20 can include one or more such central processing units 21.

[0061] The main memory 22 is called a "memory" and can store instructions and data received from the input device 24, the auxiliary memory 25, and the communication network 10 (such as the server device 30) via the input / output interface device 23, as well as the results of calculations by the central processing unit 21. The main memory 22 can include, but is not limited to, computer-readable media such as volatile memory (e.g., registers, cache, random access memory (RAM)), non-volatile memory (e.g., read-only memory (ROM), EEPROM, flash memory), and storage (e.g., hard disk drive (HDD), solid-state drive (SSD), magnetic tape, optical media). As will be readily understood, the term "computer-readable recording medium" can include media for data storage such as memory and storage, as well as transmission media such as modulated data signals, i.e., transitory signals.

[0062] The auxiliary storage device 25 is a storage device having a larger capacity than the main storage device 22. The auxiliary storage device 25 can store instructions and data (computer programs) constituting an operating system, a web browser application, a specific application for distributing and / or receiving video, etc. Furthermore, the auxiliary storage device 25 can transmit these instructions and data (computer programs) to the main storage device 22 via the input / output interface device 23 by being controlled by the central processing unit 21. The auxiliary storage device 25 can include, but is not limited to, a magnetic disk device and / or an optical disk device, etc.

[0063] The input device 24 is a device for taking in data from the outside, and may include, but is not limited to, a touch panel, a button, a keyboard, a mouse, and / or a sensor, etc. The sensor may include, but is not limited to, one or more cameras 24a, etc. and / or one or more microphones 24b, etc.

[0064] The output device 26 may include, but is not limited to, a display device, a touch panel, and / or a printer device.

[0065] In the first aspect, the terminal device 20 may be a terminal device 20 that generates motion data based on a first tracking method. In this case, the terminal device 20 (typically a personal computer, etc.) may have (or may be connected to) a device (such as an HMD or a VR device) that functions as the input device 24 and the output device 26 and is worn on the user's head, and at least one controller that functions as the input device 24 and is worn or held on the user's body other than the head. The at least one controller may include, for example, a right hand controller that may be held or worn on the user's right hand and a left hand controller that may be held or worn on the user's left hand, but is not limited to these.

[0066] Fig. 3 is a schematic diagram showing an example of a device that can be connected to the terminal device 20 or the like to generate motion data based on the first tracking method in the communication system 1 shown in Fig. 1. The terminal device 20 that generates motion data based on the first tracking method can include (connected to) an HMD (or VR glasses or the like) 100 worn on the user's head, a right hand controller 102 that can be held or worn on a part of the user's body other than the head (here, the user's right hand), and a left hand controller 104 that can be held or worn on a part of the user's body other than the head (here, the user's left hand), as shown in Fig. 3.

[0067] The HMD 100 can include a base 100A and a fixing unit 100B for fixing the base 100A to a user's head (not shown). The base 100A can include a display unit (not shown) arranged opposite both eyes of the user, a sensor unit (not shown) that detects the coordinates and angle of the user's head and generates operation data indicating these, and a communication unit (not shown) that communicates various data (including operation data, operation data, etc.) between the display unit, the sensor unit, the right hand controller 102, the left hand controller 104, and the terminal device 20 (the input / output interface device 23). The sensor unit can also include an eye tracking sensor (not shown) that is provided opposite the user's eyeball so as to directly detect the movement of the user's eyeball. The eye tracking sensor can be, for example, a gaze detection sensor configured to irradiate near-infrared light into the iris and detect the reflected light. Such an eye tracking sensor can generate operation data indicating the movement of the user's gaze. A user wearing such an HMD 100 can, by moving his / her eyeball, move an object displayed on the display unit (e.g., pointer 630 displayed in FIG. 15, etc.) and / or select an object displayed on the display unit (e.g., icons 620, 622, 624 displayed in FIG. 15, etc.).

[0068] The fixed part 100B may include a first fixed part 100B1 extending in a substantially U-shape with both ends fixed to the base part 100A, and a second fixed part 100B2 having one end fixed to the base part 100A and the other end fixed to the first fixed part 100B1. The first fixed part 100B1 and the second fixed part 100B2 surround the user's head and are biased toward the base part 100A, whereby the HMD 100 can be worn on the user's head.

[0069] The right hand controller 102 and the left hand controller 104 may have housings 102A and 104A that are symmetrical to each other. The right hand controller 102 may include, for example, a joystick 102B that can be operated by a user's finger and at least one button (here, for example, buttons 102C, 102D, and 102E) that can be pressed by the user's finger, provided in the housing 102A. Furthermore, the right hand controller 102 may include a sensor unit (not shown) provided inside the housing 102A and configured to detect the coordinates and angle of the user's right hand, generate operation data indicating the same, and generate operation data indicating operations performed by the user on each of the joystick 102B and the buttons 102C, 102D, and 102E, and a communication unit (not shown) that communicates various data (including operation data and operation data) between the HMD 100.

[0070] Similarly, the left hand controller 104 is provided in the housing 104A and can include, for example, a joystick 104B that can be operated by the user's finger, and at least one button (here, for example, buttons 104C, 104D, and 104E) that can be pressed by the user's finger. Furthermore, the left hand controller 104 can include a sensor unit (not shown) provided inside the housing 104A and configured to detect the coordinates and angle of the user's right hand, generate operation data indicating these, and generate operation data indicating operations performed by the user on each of the joystick 104B and the buttons 104C, 104D, and 104E, and a communication unit (not shown) that communicates various data (including operation data and operation data) between the HMD 100.

[0071] The communication unit of the HMD 100 can transmit the motion data and / or operation data generated by the sensor unit of the HMD 100, the motion data and / or operation data generated by the right hand controller 102, and the motion data and / or operation data generated by the left hand controller 104 to a terminal device 20, etc.

[0072] 2, in the second aspect, the terminal device 20 may be a terminal device 20 that generates motion data based on the second tracking method. In this case, the terminal device 20 (typically a smartphone, a tablet, a personal computer, or the like) may have at least one camera 24a (and at least one microphone 24b) functioning as an input device 24 and a touch panel device functioning as an output device 26 (or may be connected to these devices).

[0073] In such a hardware configuration, the central processing unit 21 can sequentially load the instructions and data (computer program) constituting the specific application stored in the auxiliary storage device 25 into the main storage device 22 and perform operations on the loaded instructions and data. This allows the central processing unit 21 to control the output device 26 via the input / output interface device 23, and also to transmit and receive various data to and from other devices (e.g., the server device 30 and / or other terminal devices 20) via the input / output interface device 23 and the communication network 10.

[0074] As a result, the terminal device 20 can execute various operations related to the distribution, generation and / or display of moving images (including operations described with reference to Figures 4A to 4D, 5A to 5D, 6A to 6D, 14, and 21, etc.) by executing the installed specific applications, etc.

[0075] It should be noted that the terminal device 20 may include one or more microprocessors and / or a graphics processing unit (GPU) instead of or in addition to the central processing unit 21.

[0076] 4. Hardware configuration of server device 30 Next, the configuration of the server device 30 used in the above-mentioned communication system 1 will be described with reference to Fig. 2. The hardware configuration of each server device 30 may be the same as that of each of the above-mentioned terminal devices 20. Therefore, the reference characters for the components of each server device 30 are shown in parentheses in Fig. 2.

[0077] 2, each server device 30 can include a central processing unit 31, a main memory device 32, an input / output interface device 33, an input device 34, an auxiliary memory device 35, and an output device 36. These devices are connected to each other by a data bus and / or a control bus.

[0078] The central processing unit 31, the main memory device 32, the input / output interface device 33, the input device 34, the auxiliary memory device 25 and the output device 36 may be substantially identical to the central processing unit 21, the main memory device 22, the input / output interface device 23, the input device 24, the auxiliary memory device 25 and the output device 26 included in each of the terminal devices 20 described above, respectively.

[0079] In such a hardware configuration, the central processing unit 31 can sequentially load the instructions and data (computer program) constituting the specific application stored in the auxiliary storage device 35 into the main storage device 32 and perform operations on the loaded instructions and data. This allows the central processing unit 31 to control the output device 36 via the input / output interface device 33, and also to transmit and receive various data to and from other devices (e.g., other server devices 30 and / or terminal devices 20) via the input / output interface device 33 and the communication network 10.

[0080] As a result, the server device 30 can execute various operations related to the distribution, generation and / or display of moving images (including operations described with reference to Figures 4A to 4D, 5A to 5D, 6A to 6D, 14, and 21, etc.) by executing the installed specific applications, etc.

[0081] Note that the server device 30 may include, instead of or in addition to the central processing unit 31, one or more microprocessors and / or a graphics processing unit (GPU).

[0082] The server device 30 can also function as a web server by executing the above-mentioned installed specific application. The server device 30 can transmit a web page including a computer program to each terminal device 20 (and / or other server devices 30) that has accessed the server device 30. Such terminal devices 20 (and / or other server devices 30) can also execute various operations related to the distribution, generation, and / or display of moving images (including operations to be described with reference to FIGS. 4A to 4D, 5A to 5D, 6A to 6D, 14, and 21, etc.) by executing the computer program included in the web page.

[0083] 5. Platform Next, we will explain an example of a platform realized by the communication network 1. This platform makes it possible for each distributor's terminal device and each viewer's terminal device to appropriately display videos, regardless of whether each distributor uses a terminal device 20 that generates and transmits motion data based on the first tracking method or a terminal device 20 that generates and transmits motion data based on the second tracking method.

[0084] The operations executed on this platform will be explained using the following three cases as examples. For ease of explanation, the terminal device 20 that generates and transmits operations based on the first tracking method will be referred to as a "VR terminal device," and the terminal device 20 that generates and transmits operations based on the second tracking method will be referred to as a "smartphone terminal device." However, even a smartphone or tablet can become a "VR terminal device" by using multiple tracking devices (for example, the devices shown in FIG. 3) connected to it.

[0085] First case (FIGS. 4A-4D): A first distributor using a VR terminal device and a second distributor using a smartphone terminal device provide videos to each viewer in cooperation. After this, one viewer (viewer A) uses a VR terminal device or a smartphone terminal device to provide videos to each viewer in cooperation with the first and second distributors. Second Case (FIGS. 5A-5D): The first distributor using a VR terminal device provides a video to each viewer alone. After that, viewer A uses a VR terminal device or a smartphone terminal device to provide a video to each viewer in cooperation with the first distributor. Third case (FIGS. 6A to 6D): The second distributor using a smartphone terminal device provides videos to each viewer alone. After that, viewer A uses a VR terminal device or a smartphone terminal device to provide videos to each viewer in cooperation with the second distributor.

[0086] 5-1. Case 1 First, the first case will be described with reference to Fig. 4A to Fig. 4D. Fig. 4A to Fig. 4D are schematic diagrams for explaining the first case regarding provision of moving images using the platform realized by the communication system 1 shown in Fig. 1. For convenience, the communication network 10 intervening between each terminal device 20 and the server device 30 is omitted in Fig. 4A to Fig. 4D.

[0087] (1) About Figure 4A 4A, a first distributor using a VR terminal device 20A and a second distributor using a smartphone terminal device 20B are jointly providing videos to the terminal devices 20 of multiple viewers, including the terminal device 20C of the viewer A. However, for the sake of simplicity of the drawings, only the terminal device 20C of the viewer A, which is a VR terminal device, is shown as the terminal devices 20 of the multiple viewers in FIGS. 4A to 4D.

[0088] The terminal device 20A of the first distributor can generate motion data (first motion data) based on the motion of the first distributor using the first tracking method for each unit time or at any timing. The length of the unit time here can be fixed or variable. This first motion data can include motion data indicating coordinates and angles (orientations) (e.g., associated with a timestamp) generated by each tracking device included in the multiple tracking devices attached to the body of the first distributor.

[0089] For example, when the terminal device 20A uses the tracking device illustrated in FIG. 3, the first operation data may include, but is not limited to, the following exemplary data. Motion data indicating the coordinates and angle of the head, generated by the HMD 100 (the sensor unit mounted on the HMD 100 has a gyro sensor, an acceleration sensor, and / or a geomagnetic sensor, and is therefore capable of generating motion data indicating the coordinates and angle of the head.) Motion data indicating the coordinates and angle of the right hand generated by the right hand controller 102 (the above-mentioned sensor unit mounted on the right hand controller 102 has a gyro sensor, an acceleration sensor, and / or a geomagnetic sensor, and is therefore capable of generating motion data indicating the coordinates and angle of the right hand.) Motion data indicating the coordinates and angle of the left hand generated by the left hand controller 104 (the above-mentioned sensor unit mounted on the left hand controller 104 has a gyro sensor, an acceleration sensor, and / or a geomagnetic sensor, and is therefore capable of generating motion data indicating the coordinates and angle of the left hand.) Operation data generated by the right hand controller 102, which indicates the content of an operation performed on a joystick or the like with the right hand Operation data generated by the left hand controller 104, which indicates the content of an operation performed on a joystick or the like with the left hand

[0090] Furthermore, the terminal device 20A of the first distributor can also generate voice data (first voice data) based on the speech of the first distributor (for example, using the microphone 24b).

[0091] The terminal device 20A can continue to transmit the first action data (and optionally the first voice data) thus generated to the server device 30 at unit time intervals or at any arbitrary timing.

[0092] On the other hand, the terminal device 20B of the second distributor can generate motion data (second motion data) for each unit time or at any timing based on the motion of the second distributor using the second tracking method. The length of the unit time here can be fixed or variable. This second motion data can include motion data indicating the depth of each of a plurality of parts of the face and the coordinates and angle of the head (e.g., associated with a timestamp), which are generated by image processing using at least one sensor (camera 24a, etc.). For example, a dot projector emits an infrared laser forming a dot (point) pattern onto the user's face, and the near-infrared camera 24a captures the infrared dots projected and reflected on the user's face, and generates an image of the infrared dots thus captured. The terminal device 20B can compare an image of a dot pattern emitted by a dot projector registered in advance with an image captured by a near-infrared camera, and calculate the depth of each point (feature point) (e.g., each of 51 points) in both images (feature point) (distance between each point and the near-infrared camera) using the positional deviation of each point (feature point). The terminal device 20B can generate motion data indicating the depth calculated in this manner. Furthermore, the terminal device 20B has a gyro sensor, an acceleration sensor, and / or a geomagnetic sensor, and can generate motion data indicating the coordinates and angle of the head.

[0093] Furthermore, the terminal device 20B can also generate voice data (second voice data) based on the speech of the second distributor (for example, by using the microphone 24b).

[0094] The terminal device 20B can continue to transmit the second action data (and optionally the second voice data) thus generated to the server device 30 at unit time intervals or at any arbitrary timing.

[0095] The server device 30 can continue to transmit "one set of action data" including the first action data (and optionally the first voice data) from the terminal device 20A and the second action data (and optionally the second voice data) from the terminal device 20B to each of the terminal device 20A, the terminal device 20B, and the terminal device 20 of viewer A (as well as to the terminal devices 20 of the other viewers not shown). In this sense, it can be said that "one set of action data" is data commonly transmitted to the terminal device 20A, the terminal device 20B, and the terminal device 20 of viewer A (as well as the terminal devices 20 of the other viewers not shown).

[0096] The server device 30 can also add, for example, the following exemplary data to the "set of motion data" and transmit it. (A) Data identifying the terminal device 20 and / or sender that transmitted the motion data (and audio data) generated by each of the terminal devices 20 (here, terminal devices 20A and 20B) of multiple distributors. (B) Data indicating whether the motion data (and audio data) generated by each of the terminal devices 20 (here, terminal devices 20A, 20B) of multiple distributors was generated according to the first tracking method or the second tracking method.

[0097] Each terminal device 20 that receives such a "set of motion data" can identify how many distributors' generated motion data (and audio data) are included in the "set of motion data" by referring to data (A). Furthermore, each terminal device 20 can identify which part of the "set of motion data" contains the motion data (and audio data) generated by the terminal device of each distributor.

[0098] Moreover, each terminal device 20 can identify whether the motion data (and voice data) generated by the terminal device of each distributor has been generated according to the first tracking method or the second tracking method by referring to the data (B). Each terminal device 20 recognizes in advance what data the motion data (and voice data) generated according to each tracking method includes. For example, each terminal device 20 recognizes in advance that the motion data generated according to the first tracking method includes data indicating the coordinates and angles of the corresponding parts generated by each tracking device (and operation data indicating the content of the operation performed by the distributor on the joystick button, etc.). Similarly, each terminal device 20 recognizes in advance that the motion data generated according to the second tracking method includes data indicating the depth of each of the multiple parts of the face and the coordinates and angle of the head. Therefore, each terminal device 20 can recognize and use the content of the motion data generated according to any tracking method by the terminal device 20 of each distributor by referring to the data (B).

[0099] In order to enable each terminal device 20 to extract and utilize the action data generated by each distributor's terminal device from the "one set of action data" received, the server device 30 can add any other data to any position of the "one set of action data".

[0100] Terminal device 20A, terminal device 20B, and terminal device 20C (terminal devices 20 of the other viewers) can generate a video based on the received "one set of motion data." In the following description, each terminal device 20 can receive and store drawing data for drawing an avatar used by each distributor in association with avatar identification data that identifies the avatar, from server device 30, in advance or at any timing. Therefore, each terminal device 20 can identify the avatar used by the distributor by identifying the distributor, and generate and display an animation of the avatar.

[0101] First, attention is focused on the terminal device 20A. The terminal device 20A can generate a moving image including an animation of an avatar (first avatar) of a first distributor based on the first motion data and an animation of an avatar (second avatar) of a second distributor based on the second motion data. Specifically, since the terminal device 20A is a VR terminal device, it can generate two types of moving images, a first moving image and a second moving image. The terminal device 20A can extract each of the first motion data and the second motion data from one set of motion data received from the server device 30 by the above-mentioned method and use them to generate a moving image.

[0102] The first video may be a video in which the first avatar and the second avatar are placed in a first virtual space (e.g., a three-dimensional virtual space) and drawn (expressed) as viewed from the "perspective of the first distributor." The second video may be a video in which the first avatar and the second avatar are placed in the first virtual space and drawn (expressed) as viewed from a certain viewpoint in the first virtual space (e.g., a virtual camera placed in the first virtual space). Furthermore, this second video may be a video drawn (expressed) as viewed from a "viewpoint common to all viewers." In other words, the certain viewpoint may be a viewpoint common to all viewers.

[0103] In another embodiment, the second video may be a video drawn (expressed) from a "common viewpoint of each viewer and each distributor." In other words, the certain viewpoint may be a common viewpoint for each viewer and each distributor.

[0104] Fig. 7 is a diagram showing an example of a first video displayed by a terminal device of a first distributor in the communication system 1 shown in Fig. 1. As shown in Fig. 7, the first video 200 displayed by the terminal device 20A of the first distributor can include a first avatar 210 of the first distributor that moves according to the first action data, and a second avatar 220 of the second distributor that moves according to the second action data. Here, the first video 200 does not include an avatar (third avatar) of viewer A.

[0105] The first video 200 may be a video depicting the first virtual space as seen from the viewpoint of the first avatar 210 placed in the first virtual space (i.e., the viewpoint of the first broadcaster). The current position of the first avatar 210 in the first virtual space and the angle at which the first avatar 210 faces may be determined and changed according to the first action data included in one set of video data received from the server device 30 (or the first action data generated and temporarily stored by the terminal device 20A).

[0106] For example, when the first broadcaster rotates his / her head 15 degrees to the right (or left), the terminal device 20A can include data indicating the angle of the head rotated 15 degrees to the right (or left) in the motion data. As a result, the terminal device 20A updates the angle of the head of the first avatar 210, so that the angle at which the first avatar 210 faces is rotated 15 degrees to the right (or left) in the first video 200. Also, when the first broadcaster lifts the right hand controller 102 and the left hand controller 104 up to his / her chest, the terminal device 20A can include data indicating the changed coordinates of the right hand and the left hand in the motion data. As a result, the terminal device 20A updates the coordinates of the right hand and the left hand of the first avatar 210, so that the right hand 210a and the left hand 210b of the first avatar 210 are drawn lifted in the first video 200, as illustrated in FIG. 7. Furthermore, when the first broadcaster presses button 102E of the right hand controller 102 (or button 104E of the left controller 104), the terminal device 20A can include data indicating the coordinates of the head that has been moved forward (or updated) in the video data. As a result, the terminal device 20A updates the coordinates at which the first avatar 210 is located, so that the viewpoint of the first avatar 210 moves forward (or backward) in the first video 200. Accordingly, the second avatar 220 is drawn larger (or smaller) in the first video 200.

[0107] In the first video 200, the second avatar 220 moves according to the second motion data in the same way as the first avatar 210 moves according to the first motion data. However, since the terminal device 20B of the second distributor is a smartphone terminal device, the second motion data generated by the terminal device 20B is motion data generated based on the second tracking method (motion data indicating the depth of each of a plurality of parts of the face and the coordinates and angle of the head). Therefore, in the first video 200, the head of the second avatar 220 moves (for example, shakes the head from side to side) based on the second motion data (for example, data indicating the coordinates and angle of the head). In addition, the facial expression of the second avatar 220 changes based on the second motion data (data indicating the depth of each of a plurality of parts of the face). Furthermore, the direction in which the second avatar 220 faces changes based on the second motion data (data indicating the coordinates and angle of the head). However, the position of the second avatar 220 does not change.

[0108] The object 245 illustrated in FIG. 7 will be described later in "5-4. Modifications."

[0109] Fig. 8 is a diagram showing an example of a second video displayed by a terminal device of a first distributor in the communication system 1 shown in Fig. 1. As illustrated in Fig. 8, a second video 300 includes a first avatar 210 of a first distributor that moves according to the first motion data and a second avatar 220 of a second distributor that moves according to the second motion data, but does not include an avatar (third avatar) of viewer A.

[0110] The second video 300 is a video that depicts the first virtual space from a viewpoint common to all of the multiple viewers (including viewer A). The common viewpoint here may be a viewpoint seen from any point (such as a virtual camera installed at this point) in the first virtual space described above. For simplification, FIG. 8 shows an example in which the second video 300 is depicted with both the first avatar 210 and the second avatar 220 viewed from the front, but depending on the position of the viewpoint placed in the first virtual space, the first avatar 210 and / or the second avatar 220 may face to the side or backward in the second video 300. In one embodiment, the second video 300 may be generated such that both the first avatar 210 and the second avatar 220 face forward, regardless of the position of the viewpoint placed in the first virtual space.

[0111] Since the first motion data is generated according to a first tracking method, the position of the first avatar 210, which moves based on this first motion data, in the first virtual space may change, and the direction in which the first avatar 210 faces may also change. Therefore, in the second video 300, the position of the first avatar 210 may change, and the direction in which the first avatar 210 faces may also change. In contrast, since the second motion data is generated according to a second tracking method, the coordinates in the first virtual space of the second avatar 220, which moves based on this second motion data, do not change. Therefore, in the second video 300, the position of the second avatar 220 does not change (however, in the second video 300, the facial expression, etc. of the second avatar 220 may change based on the second motion data).

[0112] For this reason, in one embodiment, as illustrated in FIG. 8, the second avatar 220 whose position in the first virtual space is fixed may be placed in the front row, and the first avatar 210 whose position in the first virtual space changes may be placed in the back row behind the front row. Furthermore, in one embodiment, as illustrated in FIG. 8, the first avatar 210 placed in the back row may be placed at a higher position than the second avatar 220 placed in the front row. In this way, by placing multiple avatars in a row corresponding to their characteristics among multiple rows (front row and back row), each of these multiple avatars can be made easy to see for each user (first broadcaster, second broadcaster, each viewer). Note that in another embodiment, it is also possible to place the first avatar 210 in the front row and the second avatar 220 in the back row, inversely to the above.

[0113] In one embodiment, the terminal device 20A can display the first moving image 200 described above with reference to FIG. 7 and the second moving image 300 described above with reference to FIG. 8 in mutually different regions in a display unit (such as the HMD 100) that is the output device 26. For example, the terminal device 20A can display the first moving image 200 in a part of a specific region included in a display region formed by the display unit, such as the right, left, upper or lower part, and display the second moving image 300 in the remaining region of the specific region. In another preferred embodiment, the terminal device 20A can also display the second moving image 300 inside the region in which the first moving image 200 is displayed, as exemplified in FIG. 9.

[0114] FIG. 9 is a diagram showing another example of the first video and the second video displayed by the terminal device of the first broadcaster in the communication system 1 shown in FIG. 1. As shown in FIG. 9, the terminal device 20A can combine the second video 300 with the first video 200 and display it, for example, inside the area in which the first video 200 is displayed. Specifically, the terminal device 20A can movably display the second video 300 in conjunction with a part (here, the left hand 210b) of the first avatar 210 included in the first video 200. In this case, when the first broadcaster moves his / her left hand holding the left hand controller 104, the left hand 210b of the first avatar 210 included in the first video 200 moves accordingly. In conjunction with such left hand 210b, the second video 300 held (drawn as if held) by the left hand 210b can also move. In one embodiment, the second video 200 can be displayed as a terminal device (pseudo terminal device) pseudo-displayed in the first video 200.

[0115] Returning to FIG. 4A, attention is now focused on terminal device 20B. The terminal device 20B can generate a moving image including an animation of the avatar (first avatar) of the first distributor based on the first motion data and an animation of the avatar (second avatar) of the second distributor based on the second motion data. Specifically, since the terminal device 20A is a smartphone terminal device, it can generate a second moving image. Note that the terminal device 20B can extract each of the first motion data and the second motion data from one set of motion data received from the server device 30 by the above-mentioned method and use them to generate a moving image.

[0116] As described above, the second video may be a video in which the first avatar and the second avatar are arranged in the first virtual space and drawn (expressed) as viewed from a certain viewpoint in the first virtual space (for example, a virtual camera arranged in the first virtual space). Furthermore, this second video may be a video drawn (expressed) as viewed from a "viewpoint common to all viewers". Here, the "one set of motion data" that the terminal device 20B receives from the server device 30 is the same as the "one set of motion data" that the terminal device 20A receives from the server device 30. In addition, the method by which the terminal device 20B generates the second video is the same as the above-mentioned method by which the terminal device 20A generates the second video. As a result, the second video generated by the terminal device 20B may be substantially the same as the second video 300 (see FIG. 8) generated by the terminal device 20A. Therefore, the terminal device 20B can display the second video 300 as exemplified in FIG. 8 on the output device 26, such as a touch panel or a display.

[0117] Referring again to FIG. 4A, attention is now focused on terminal device 20C. The terminal device 20C can generate a moving image including an animation of an avatar (first avatar) of a first distributor based on the first motion data and an animation of an avatar (second avatar) of a second distributor based on the second motion data. Specifically, since the terminal device 20C is a VR terminal device, it can generate two types of moving images, a first moving image and a second moving image, similar to the above-mentioned terminal device 20A. Note that the terminal device 20A can extract each of the first motion data and the second motion data from one set of motion data received from the server device 30 by the above-mentioned method, and use them to generate a moving image.

[0118] The first video may be a video in which a first avatar and a second avatar are placed in a first virtual space (eg, a three-dimensional virtual space) and are rendered (expressed) as seen from the "perspective of viewer A."

[0119] Fig. 10 is a diagram showing an example of a first moving image displayed by the terminal device of viewer A in the communication system 1 shown in Fig. 1. The first moving image 400 displayed by the terminal device 20C of viewer A can include a first avatar 210 of the first distributor that moves according to the first action data, and a second avatar 220 of the second distributor that moves according to the second action data, as shown in Fig. 10. Note that the first moving image 400 does not display the avatar of viewer A (third avatar).

[0120] Since the terminal device 20C is a VR terminal device, it can generate motion data (third motion data) using the first tracking method based on the motion of the viewer A who is the user of the terminal device 20C.

[0121] The first video 400 may be a video depicting the first virtual space as seen from the viewpoint of the third avatar of viewer A placed in the first virtual space (i.e., the viewpoint of viewer A). The current position of the third avatar in the first virtual space and the angle at which the third avatar faces, etc. (i.e., the viewpoint of the third avatar or viewer A used in the first video 400) may be determined and changed according to the third action data. This is similar to the first video 200 generated by the terminal device 20A described above.

[0122] For example, when viewer A rotates his / her head 15 degrees to the right (or left), the angle at which the third avatar faces (and thus the viewpoint of the third avatar) is rotated 15 degrees to the right (or left) in the first video 400. When viewer A raises the right hand controller 102 and the left hand controller 104 up to his / her chest, the right hand and left hand of the third avatar are raised and depicted in the first video 400, as described above. Furthermore, when viewer A presses the button 102E of the right hand controller 102 (or the button 104E of the left controller 104), the viewpoint of the third avatar moves forward (or backward), and therefore the first avatar 210 and the second avatar 220 are depicted larger (or smaller) in the first video 400.

[0123] In the first video 400, the first avatar 210 moves according to the first motion data. Since the terminal device 20A of the first distributor is a VR terminal device, the first motion data generated by the terminal device 20A is motion data generated based on the first tracking method. Therefore, in the first video 400, the position of the first avatar 210, the direction in which the first avatar 210 faces, etc. may change based on the first motion data.

[0124] In contrast to the first moving image 200 (FIG. 7) generated by the terminal device 20A, the first moving image 400 (FIG. 10) generated by the terminal device 20C may display the entire body of the first avatar 210 of the first distributor. This is because the first virtual space is depicted from the viewpoint of the viewer A (third avatar) in the first moving image 400.

[0125] Also, in the first video 400, the second avatar 220 moves according to the second motion data. However, since the terminal device 20B of the second distributor is a smartphone terminal device, the second motion data generated by the terminal device 20B is motion data generated based on the second tracking method (motion data indicating the depth of each of multiple parts of the face and the coordinates and angle of the head). Therefore, in the first video 400, the head of the second avatar 220 moves (for example, shaking the head from side to side) based on the second motion data, but the position of the second avatar 220 does not change.

[0126] As described above, the second video may be a video in which the first avatar and the second avatar are arranged in the first virtual space and drawn (expressed) from a certain viewpoint in the first virtual space. Furthermore, the second video may be a video drawn (expressed) from a "viewpoint common to each viewer". Here, the "one set of motion data" that the terminal device 20C receives from the server device 30 is the same as the "one set of motion data" that the terminal devices 20A and 20B receive from the server device 30. In addition, the method by which the terminal device 20C generates the second video is the same as the above-mentioned method by which the terminal devices 20A and 20B generate the second video. As a result, the second video generated by the terminal device 20C may be substantially the same as the second video 300 (see FIG. 8) generated by the terminal devices 20A and 20B. Therefore, the terminal device 20C can display the second video 300 as exemplified in FIG. 8 on the output device 26, such as the HMD 100.

[0127] Similarly to the terminal device 20A described above, the terminal device 20C can also combine and display the first moving image 400 and the second moving image 300. The technique described above with reference to Fig. 9 in relation to the terminal device 20A can be similarly applied to the first moving image 400 and the second moving image 300 displayed by the terminal device 20C.

[0128] (2) Regarding Figure 4B Referring to Fig. 4B, a first distributor using a VR terminal device 20A and a second distributor using a smartphone terminal device 20B are working together to provide videos to the terminal devices 20 of multiple viewers, including the terminal device 20C of the viewer A. The only difference between Fig. 4B and Fig. 4A is that the viewer A uses the terminal device 20D, which is a smartphone terminal device, instead of the terminal device 20C (Fig. 4A), which is a VR terminal device.

[0129] The operations performed by terminal device 20A (including displaying two types of moving images) and the operations performed by terminal device 20B (including displaying one type of moving image) are the same as those described above in relation to FIG. 4A.

[0130] The terminal device 20D can generate a moving image including an animation of an avatar (first avatar) of a first distributor based on the first motion data and an animation of an avatar (second avatar) of a second distributor based on the second motion data. Specifically, since the terminal device 20D is a smartphone terminal device, it can generate a second moving image. Note that the terminal device 20D can extract each of the first motion data and the second motion data from one set of motion data received from the server device 30 by the above-mentioned method and use them to generate a moving image.

[0131] Unlike the terminal device 20C shown in FIG. 4A, the terminal device 20D can display the second video instead of the first video. As described above, the second video may be a video in which the first avatar and the second avatar are arranged in the first virtual space and drawn (expressed) from a certain viewpoint in the first virtual space. Furthermore, the second video may be a video drawn (expressed) from a "viewpoint common to each viewer". Here, the "one set of motion data" that the terminal device 20D receives from the server device 30 is the same as the "one set of motion data" that the terminal device 20A receives from the server device 30. In addition, the method of generating the second video by the terminal device 20D is the same as the method of generating the second video by the terminal device 20A described above. As a result, the second video generated by the terminal device 20D may be substantially the same as the second video 300 (see FIG. 8) generated by the terminal device 20A. Therefore, the terminal device 20D can display the second video 300 as exemplified in FIG. 8 on the output device 26, such as a touch panel or a display.

[0132] (3) Regarding Figure 4C FIG. 4C illustrates a transition from the situation illustrated in FIG. 4A to a situation in which viewer A, who is a user of terminal device 20C, provides a video as a third distributor in cooperation with the first and second distributors.

[0133] 4C differs from FIG. 4A in that the terminal device 20C, which is a VR terminal device, transmits third motion data generated using the first tracking method based on the motion of the third distributor to the server device 30. As a result, the server device 30 continues to transmit one set of motion data including the first motion data, the second motion data, and the third motion data to the terminal device 20A, the terminal device 20B, the terminal device 20C, and the terminal device 20 of each viewer (not shown).

[0134] Let us focus on the terminal device 20A. The terminal device 20A can generate a moving image including an animation of the first distributor's avatar (first avatar) based on the first motion data, an animation of the second distributor's avatar (second avatar) based on the second motion data, and an animation of the third distributor's avatar (third avatar) based on the third motion data. Specifically, since the terminal device 20A is a VR terminal device, it can generate two types of moving images, a first moving image and a second moving image. Note that the terminal device 20A can extract each of the first motion data, the second motion data, and the third motion data from one set of motion data received from the server device 30 by the above-mentioned method, and use them to generate a moving image.

[0135] Fig. 11 is a diagram showing another example of the first video displayed by the terminal device of the first distributor in the communication system 1 shown in Fig. 1. As shown in Fig. 11, the first video 200A displayed by the terminal device 20A of the first distributor includes a first avatar 210 of the first distributor that moves according to the first motion data, a second avatar 220 of the second distributor that moves according to the second motion data, and a third avatar 450 that moves according to the third motion data.

[0136] The method of generating or displaying second avatar 220 in first moving image 200A is similar to that in first moving image 200, and therefore will not be described here.

[0137] In the first video 200A, the third avatar 450 moves according to the third motion data. Since the terminal device 20C of the third distributor is a VR terminal device, the third motion data generated by the terminal device 20C is motion data generated based on the first tracking method. Therefore, in the first video 200A, the position of the third avatar 450 and the direction in which the third avatar 450 faces may change based on the third motion data.

[0138] Fig. 12 is a diagram showing another example of the second video displayed by the terminal device of the first distributor in the communication system 1 shown in Fig. 1. As illustrated in Fig. 12, the second video 300A may include a first avatar 210 of the first distributor that operates according to the first motion data, a second avatar 220 of the second distributor that operates according to the second motion data, and a third avatar 450 of the third distributor that operates according to the third motion data.

[0139] Second moving image 300A differs from second moving image 300 shown in FIG. 8 in that second moving image 300A includes third avatar 450 (other aspects are as described above in relation to second moving image 300).

[0140] Because the third motion data is generated according to the first tracking method, the position of third avatar 450 moving based on this third motion data in the first virtual space may change, and the direction in which third avatar 450 faces may also change. Therefore, in second video 300A, the position of third avatar 450 may change, and the direction in which third avatar 450 faces, etc. may also change.

[0141] Additionally, the third avatar 450 may preferably be placed in the back row together with the first avatar 210 for the reasons described above in relation to the second video 300.

[0142] Returning to FIG. 4C, attention is now focused on terminal device 20B. The terminal device 20B is a smartphone terminal device and is therefore capable of generating a second moving image. As described above, the second moving image generated by terminal device 20B is substantially the same as the second moving image generated by terminal device 20A (FIG. 12).

[0143] Next, attention is focused on the terminal device 20C. The terminal device 20C can generate a moving image including an animation of the first distributor's avatar (first avatar) based on the first motion data, an animation of the second distributor's avatar (second avatar) based on the second motion data, and an animation of the second distributor's avatar (third avatar) based on the third motion data. Specifically, since the terminal device 20C is a VR terminal device, it can generate two types of moving images, a first moving image and a second moving image. Note that the terminal device 20C can extract each of the first motion data, the second motion data, and the third motion data from one set of motion data received from the server device 30 by the above-mentioned method, and use them to generate a moving image.

[0144] FIG. 13 is a diagram showing an example of a first video displayed by a terminal device of a third distributor in the communication system 1 shown in FIG. 1. The first video 400A generated by the terminal device 20C is different from the first video 400 (FIG. 10) generated by the terminal device 20C described above in that the first video 400A includes a third avatar 450 that moves according to the third action data. FIG. 13 shows an example in which a part of the third avatar 450 (left hand 450a) is displayed in the first video 400A. As the third distributor lifts his left hand holding the left hand controller 104, the left hand 450a is lifted and drawn in the first video 400A, as exemplified in FIG. 13. Note that the viewpoint of the third avatar 450 or the third distributor in the first video 400A can change according to the third action data, as described above in relation to the first video 400.

[0145] For the reasons described above, the second moving image displayed on terminal device 20C is substantially the same as the second moving image (FIG. 12) displayed on terminal device 20A and terminal device 20B.

[0146] (4) Regarding Figure 4D FIG. 4D illustrates a transition from the situation illustrated in FIG. 4B to a situation in which viewer A, who is a user of terminal device 20C, provides a video as a third distributor in cooperation with the first and second distributors.

[0147] 4D differs from FIG. 4B in that terminal device 20D, which is a smartphone terminal device, transmits fourth action data generated using the second tracking method based on the action of the third distributor to server device 30. As a result, server device 30 continues to transmit one set of action data including the first action data, the second action data, and the fourth action data to terminal device 20A, terminal device 20B, terminal device 20D, and terminal device 20 of each viewer (not shown).

[0148] Let us focus on the terminal device 20A. The terminal device 20A can generate a moving image including an animation of the first distributor's avatar (first avatar) based on the first motion data, an animation of the second distributor's avatar (second avatar) based on the second motion data, and an animation of the third distributor's avatar (third avatar) based on the fourth motion data. Specifically, since the terminal device 20A is a VR terminal device, it can generate two types of moving images, a first moving image and a second moving image. Note that the terminal device 20A can extract each of the first motion data, the second motion data, and the fourth motion data from one set of motion data received from the server device 30 by the above-mentioned method, and use them to generate a moving image.

[0149] The first moving image generated by terminal device 20A in the situation shown in Fig. 4D differs from the first moving image (Fig. 11) generated by terminal device 20A in the situation shown in Fig. 4C in that third avatar 450 moves based on the fourth movement generated in accordance with the second tracking method. Therefore, as described above, the position of third avatar 450 does not change in the first moving image. However, the head of third avatar 450 moves based on the fourth movement data (for example, the head swings from side to side, changing the direction the head faces), and the facial expression, etc. of third avatar 450 may change based on the fourth movement data.

[0150] On the other hand, the second moving image generated by terminal device 20A in the situation shown in Fig. 4D differs from the second moving image (Fig. 12) generated by terminal device 20A in the situation shown in Fig. 4C in that third avatar 450 moves based on the fourth movement generated in accordance with the second tracking method. Therefore, as described above, the position of third avatar 450 does not change in the second moving image. However, the head of third avatar 450 moves based on the fourth movement data (for example, the head swings from side to side and the direction in which the head faces also changes), and the facial expression, etc. of third avatar 450 may change based on the fourth movement data.

[0151] Next, attention is focused on the terminal device 20B. The terminal device 20B is a smartphone terminal device and is therefore capable of generating a second moving image. For the reasons described above, the second moving image generated by terminal device 20B is substantially identical to the second moving image generated by terminal device 20A in the situation shown in FIG. 4D.

[0152] Next, attention is focused on the terminal device 20D. The terminal device 20D is also a smartphone terminal device and is therefore capable of generating a second moving image. For the reasons described above, the second moving image generated by terminal device 20D is substantially identical to the second moving image generated by terminal device 20A in the situation shown in FIG. 4D.

[0153] 5-2. Case 2 In the second case, the first distributor using a VR terminal device provides a video to each viewer alone. After this, viewer A uses a VR terminal device or a smartphone terminal device to provide a video to each viewer in cooperation with the first distributor.

[0154] The second case corresponds to the above-mentioned first case (FIGS. 4A to 4D) except for the second distributor, its terminal device 20B, and the second action data. That is, FIG. 5A to FIG. 5D correspond to FIG. 4A to FIG. 4D except for the second distributor, its terminal device 20B, and the second action data. Therefore, to avoid redundant explanation, FIG. 5A to FIG. 5D only show the data (first action data to fourth action data) transmitted between the devices, and the explanation thereof will be omitted.

[0155] 5-3. Case 3 In the third case, the second distributor using a smartphone terminal device provides the video to each viewer alone. After that, viewer A uses a VR terminal device or a smartphone terminal device to provide the video to each viewer in cooperation with the second distributor.

[0156] The third case corresponds to the above-mentioned first case (FIGS. 4A to 4D) except for the first distributor, its terminal device 20A, and the first operational data. That is, FIG. 6A to FIG. 6D correspond to FIG. 4A to FIG. 4D except for the first distributor, its terminal device 20A, and the first operational data. Therefore, in order to avoid redundant explanation, FIG. 6A to FIG. 6D only show the data transmitted between the devices, and the explanation thereof will be omitted.

[0157] 5-4.Modifications In the various embodiments described above, the server device 30 transmits all of the motion data received from the terminal device 20 of each distributor as one set of motion data to the terminal device 20 of each distributor and to the terminal device 20 of each viewer. That is, in the various embodiments described above, the server device 30 transmits certain motion data, including the certain motion data in one set of motion data, to the terminal device of the distributor that transmitted the certain motion data.

[0158] In this case, as the total number of distributors who jointly distribute videos increases, the amount of motion data that each distributor's terminal device 20 receives from the server device 30, i.e., the amount of motion data that the server device 30 transmits to each distributor's terminal device 20, increases, which may result in a problem that the communication network becomes congested and it may become difficult for each distributor's terminal device 20 to smoothly provide videos to each viewer's terminal device 20 via the server device 30. To address this problem, in another embodiment, the server device 30 may adopt a configuration that does not transmit certain motion data to a terminal device of a distributor that has transmitted the certain motion data. This can reduce the amount of motion data that each distributor's terminal device 20 receives from the server device 30, i.e., the amount of motion data that the server device 30 transmits to the terminal device 20 of each distributor, and therefore videos can be more smoothly provided from each distributor's terminal device 20 to each viewer's terminal device 20 via the server device 30. In this case, the terminal device of this distributor can generate a video using the certain motion data that it has generated. For example, in the configuration shown in Figures 4A to 4D, the server device 30 can transmit the first action data only to the terminal device 20B and each viewer's terminal device 20 but not to the terminal device 20A, and similarly transmit the second action data only to the terminal device 20A and each viewer's terminal device 20 but not to the terminal device 20B. Also, in the configuration shown in Figure 4C, the server device 30 can transmit the third action data only to the terminal device 20C but not to the fourth action data.

[0159] In the various embodiments described above, when the viewer A using the terminal device 20C, which is a VR terminal device, watches the first video provided by the terminal device 20 of each distributor (for example, the situations shown in each of FIG. 4A, FIG. 5A, and FIG. 6A), the third avatar 450 of the viewer A is not displayed not only in the first video displayed by the terminal device 20C of the viewer A, but also in the first video displayed by the terminal device 20 of each distributor. That is, each distributor can recognize where the avatar of each distributor is located in the first video, but cannot recognize where the avatar of each viewer is located. Therefore, for example, even if a situation occurs in which the avatar of the viewer is too close to the distributor's avatar in the first video, a problem may occur in which the distributor cannot recognize such a situation at all. Furthermore, a problem may occur in which the avatar of the distributor and the avatar of the viewer are too close to each other and sink into each other, which impairs the sense of reality or creates a sense of discomfort.

[0160] Therefore, in one embodiment, the viewer's terminal device 20 can determine whether or not the distance between any target avatar among the avatars of at least one distributor and the viewer's avatar in the first virtual space is equal to or less than a threshold value. When the viewer's terminal device 20 determines that the distance is equal to or less than the threshold value, and the motion data relating to the viewer's motion generated by the viewer's terminal device 20 indicates a motion of moving toward the target avatar (for example, the motion data indicates a motion of moving in a direction that further reduces the distance between the coordinates of the target avatar and the coordinates of the viewer's avatar), the viewer's terminal device 20 can invalidate such motion data (not use it for generating or displaying the first video). As a result, in the first video, the viewer's avatar cannot move into an area less than the threshold value with the target avatar as the center, so the distributor can participate in providing the video to the viewer with a sense of security that the above-mentioned situation will not occur. Furthermore, by preventing the occurrence of a situation in which multiple avatars are embedded into each other, it is possible to address the problem of a loss of realism or a sense of incongruity.

[0161] The occurrence of the above situation can be at least partially suppressed by using the display method described below in combination with or separately from such a method of invalidating the motion data. Specifically, when the distance between the broadcaster's own avatar and the viewer's avatar in the first virtual space is equal to or less than a threshold, the terminal device of a certain broadcaster can display an arbitrary object (such as object 245 illustrated in FIG. 7) indicating the presence of the viewer's avatar at the position of the viewer's avatar (which is not displayed in the first video as described above) in the first video. The arbitrary object can be text and / or an image, and can also include information about the viewer's avatar (the viewer's handle name and / or an image using a part of the viewer's avatar). In order to achieve this, the terminal device of the certain broadcaster needs to recognize the coordinates of the viewer's avatar in the first virtual space, and can therefore receive, from the terminal device of the viewer, the coordinates of the viewer's avatar, or motion data about the viewer's motion for calculating the coordinates, etc.

[0162] The above-mentioned method of invalidating the motion data can be applied not only to the terminal device 20 of the viewer but also to the terminal device 20 of the distributor. For example, the terminal device 20 of a certain distributor can determine whether or not the distance between a target avatar among the avatars of at least one other distributor and the avatar of the certain distributor is equal to or less than a threshold in the first virtual space. When the terminal device 20 of the certain distributor determines that the distance is equal to or less than the threshold, and the motion data related to the motion of the certain distributor generated by the terminal device 20 of the certain distributor indicates a motion of moving toward the target avatar (for example, the motion data indicates a motion of moving in a direction that further reduces the distance between the coordinates of the target avatar and the coordinates of the avatar of the certain distributor), the terminal device 20 of the certain distributor can invalidate such motion data (not use it for generating or displaying the first video). As a result, in the first video, the avatar of the certain distributor cannot move into an area less than the threshold centered on the target avatar. Although each distributor can recognize where the other distributors' avatars are located, each distributor can cooperate with other distributors in providing videos to viewers while having the peace of mind that the other distributors' avatars will not approach his / her own avatar abnormally.

[0163] In addition, in the various embodiments described above, it has been described that the server device 30 transmits "one set of motion data" including motion data related to the motion of each distributor to the terminal device of each distributor and the terminal device of each viewer. This "one set of motion data" may include data indicating which of the first tracking method and the second tracking method each distributor adopted to generate the motion data. In this case, by receiving the "one set of motion data", the terminal device of each distributor and / or the terminal device of each viewer can combine the first video and / or the second video with the tracking method used by each distributor when displaying the first video and / or the second video and display it on the display unit.

[0164] Specifically, as a first example, focusing on Figure 16, which will be referred to later, Figure 16 shows an example in which a terminal device 20C, which is a VR terminal device, displays an input object 710, which will be described later, in combination with a first video 700 that operates based on motion data regarding the motion of user A.

[0165] The terminal device 20C can display six icons each showing the handle names "Mr. U", "Mr. V", "Mr. W", "Mr. X", "Mr. Y" and "Mr. Z". User A can determine the broadcaster to be viewed by moving the pointer to an icon corresponding to a desired one of these broadcasters and selecting it. The terminal device 20C can also display, in association with each broadcaster, an image showing the broadcaster's avatar (e.g., a profile image), the name of the content distributed by the broadcaster, and / or the details of the content distributed by the broadcaster. In this case, the terminal device 20C can receive information to be displayed in this way from the server device 30.

[0166] The terminal device 20C can display information indicating whether the first tracking method or the second tracking method is used to distribute a video, in association with the icon of each distributor. Here, as an example, the terminal device 20C can display information "VR" in association with a distributor using the first tracking method, and information "smartphone" in association with a distributor using the second tracking method.

[0167] By displaying such information on the terminal device 20C, each viewer including user A can refer to the above information to decide which distributor to select when he / she has a desire to "share video distribution with a distributor using a VR terminal device" or "share video distribution with a distributor using a smartphone terminal device." Furthermore, when each viewer considers sharing a video with a distributor, taking into consideration whether the terminal device he / she uses is a VR terminal device with higher drawing performance (or a smartphone terminal device with lower drawing performance) and / or the stability of the current communication network, if he / she wishes to share a video with a lower load, he / she can select a distributor using a smartphone terminal device (such as "Mr. U," "Mr. V," or "Mr. W"), and if he / she wishes to enjoy a video distribution in which users move around each other in a virtual space, he / she can select a distributor using a VR terminal device (such as "Mr. X," "Mr. Y," or "Mr. Z"). As a result, it is possible to prevent a situation in which a communication network becomes congested when multiple distributors share videos.

[0168] Note that, while FIG. 16 illustrates an example of a screen displayed by terminal device 20C, which is a VR terminal device, information indicating the tracking method used by each broadcaster (such as "VR" or "smartphone") may also be displayed in a similar manner on a screen displayed by a smartphone terminal device.

[0169] As a second example, attention is focused on FIG. 10 referred to above. In FIG. 10, a first video 400 displayed by the terminal device 20C of the viewer A is illustrated. As shown in FIG. 10, the terminal device 20C can display information indicating the tracking method used by each distributor in association with the avatar of the distributor displayed in the first video 400. For example, the terminal device 20C can display information 210A of "VR" indicating the first tracking method used by the first distributor in association with the first avatar 210 of the first distributor (for example, below the first avatar 210), and display information 220A of "smartphone" indicating the second tracking method used by the second distributor in association with the second avatar 220 of the second distributor (for example, below the second avatar 220).

[0170] By displaying such information on the terminal device 20C, each viewer including viewer A can select a gift suitable for the tracking method used by a broadcaster from among multiple gifts when selecting a gift to send to that broadcaster. For example, since the first broadcaster uses the first tracking method, each viewer can recognize that the first avatar 210 of the first broadcaster can wear a (more complicated) gift that can be held and used or operated, such as a "microphone," "tambourine," or "gun" that is held in the hand. Thus, each viewer can send to the first broadcaster a more complicated gift such as a "microphone" suitable for the first tracking method. On the other hand, since the second broadcaster uses the second tracking method, each viewer can recognize that it is preferable that the second avatar 220 of the second broadcaster wears a simpler gift. Thus, each viewer can send to the second broadcaster a (simpler) gift that the avatar simply wears on its body, such as a "hat," "ribbon," or "beard" suitable for the second tracking method.

[0171] As a third example, a situation may be considered in which user A participates in a joint distribution. Fig. 28 is a diagram showing another example of a screen displayed when the terminal device in the communication system shown in Fig. 1 executes the "joint distribution mode."

[0172] The terminal device 20C can display, for example, "collaboration broadcast I" and "collaboration broadcast II" in the input object 710 as icons indicating joint broadcasts (collaboration broadcasts) in which the user A can participate. The terminal device 20C can display, in association with each icon, a profile image (avatar image) of each broadcaster appearing in the joint broadcast, and the contents of the joint broadcast. For example, for the icon "collaboration broadcast I," the terminal device 20C can display profile images (P1, P2, P3) corresponding to, for example, three broadcasters appearing in the joint broadcast, and the contents of the joint broadcast ("Let's do karaoke."). Furthermore, for the icon "collaboration broadcast II," the terminal device 20C can display profile images (P4, P5, P6) corresponding to, for example, three broadcasters appearing in the joint broadcast, and the contents of the joint broadcast ("Relaxed chat.").

[0173] Furthermore, the terminal device 20C can display the information "VR" in association with each broadcaster (whose profile images P1, P2, P4) who uses the first tracking method, and can display the information "smartphone" in association with each broadcaster (whose profile images P3, P5, P6) who uses the second tracking method.

[0174] User A can participate in the collaborative broadcast corresponding to a desired icon by selecting any one of the icons (in FIG. 28, by placing pointer 630 over icon "collaborative broadcast II").

[0175] As a result, User A can refer to the information such as that shown in FIG. 28 to select either a joint distribution with many distributors using the first tracking method, or a joint distribution with many distributors using the second tracking method.

[0176] In this way, the terminal device 20C can improve usability by displaying information indicating whether the first tracking method or the second tracking method is being used to distribute a video, in association with the profile image (avatar image) of each broadcaster.

[0177] In addition, although not explicitly mentioned in the various embodiments described above because it is self-evident, the terminal device of each distributor and the terminal device of each viewer can acquire voice data generated by the terminal device of each distributor that is included in the "one set of motion data" received from the server device or that is combined with the "one set of motion data." By using such voice data, the terminal device of each distributor and the terminal device of each viewer can play the first video and / or the second video that incorporates the voices emitted by each distributor, or can play the voices emitted by each sender while displaying the first video and / or the second video.

[0178] In addition, in the above-mentioned various examples, the description has been given with a focus on a method (client rendering method) in which each terminal device 20 executes operations related to the generation of the first moving image and the second moving image. However, it is also possible to adopt a method (server rendering method) in which one server device 30 alone or multiple server devices 30 share and execute the operations related to the generation of the moving images described in each of Figs. 4A to 6D as being executed by each terminal device 20. In this case, the one or more server devices 30 execute various operations related to the generation of the moving images on behalf of each terminal device 20, thereby generating the first moving image and / or the second moving image and transmitting them to the terminal device 20C. As described above, the server device 30 receives the motion data from the terminal device 20 of each distributor, and can execute the generation of various moving images using the motion data on behalf of each terminal device 20. Meanwhile, the server device 30 does not necessarily receive the motion data from the terminal device 20 of each viewer. Therefore, when the server rendering method is adopted, the server device 30 may receive the motion data from the terminal device of each viewer at unit time or at any timing.

[0179] Also, in the various examples described above, the explanation has been focused on a method (client rendering method) in which each terminal device 20 executes operations related to the generation of the first moving image, the second moving image, etc. However, each terminal device 20 can also adopt a method (browser rendering method) in which the terminal device 20 receives a web page from the server device 30 or the web server device 30 via a web browser and executes a computer program included in the web page to perform operations related to the generation of the first moving image, the second moving image, etc.

[0180] According to the above-described platform, even if the terminal device of any distributor uses either the first tracking method or the second tracking method to generate motion data, each terminal device can generate or display a video according to the tracking method adopted by the terminal device of the distributor. Furthermore, each terminal device can generate or display one or both of the first video and the second video available depending on whether the terminal device is a VR terminal device or a smartphone terminal device. This enables the above-described platform to provide videos to each viewer, etc., using a tracking device worn on the distributor's body.

[0181] 6. Operations performed by the communication network 1 6-1.Basic operation Next, a specific example of an operation executed in the communication network 1 using the above-mentioned platform will be described, focusing in particular on the above-mentioned second moving image displayed by the terminal device 20. In the following, in order to avoid redundant explanation, the matters explained in relation to the above-mentioned platform will be omitted.

[0182] Fig. 14 is a flow diagram showing an example of the operation executed in the communication network 1 shown in Fig. 1. First, in step (hereinafter referred to as "ST") 500, a user (here, "user A") operates the terminal device 20C to cause the terminal device 20C to start and execute a specific application for receiving the provision of a video service. Note that the operation described below is similarly applicable to any other terminal device (VR terminal device) 20.

[0183] Next, in ST502, the terminal device 20C executing the specific application can execute a "preparation mode". This "preparation mode" is a mode in which the terminal device 20C prepares to receive or distribute a video when the terminal device 20C does not receive a video provided by another terminal device 20C, nor distributes a video to another terminal device 20. The terminal device 20C can store mode identification data indicating which mode is being executed as the current mode. The terminal device 20C can store mode identification data indicating the "preparation mode" from when it transitions to the "preparation mode" until it ends the "preparation mode". At any timing, the terminal device 20C can determine which mode is being executed by referring to the mode identification data.

[0184] When the terminal device 20C is in the "preparation mode", it can display a screen (home screen) as shown in FIG.

[0185] FIG. 15 is a diagram showing an example of a screen displayed by a terminal device when the "preparation mode" is executed in the communication system shown in FIG. 1. As shown in FIG. 15, the terminal device 20C, which is a VR terminal device, can display a first video (first type of video) 600 that operates based on motion data related to the motion of user A on the home screen. As described above in "5. Platform", the first video 600 is drawn from the viewpoint of user A, and the viewpoint can change based on the motion data. In addition, the first video 600 can display an avatar 450 of user A (here, the left hand 450a of the avatar 450, etc.). The avatar 450 displayed in the first video 600 also operates according to the motion data indicating the motion of user A.

[0186] Furthermore, the terminal device 20C can display an input object 610 in combination with the first video 600. The input object 610 can function as a user interface that enables operation input by the user A. Specifically, the input object 610 can include a plurality of icons, for example, an icon 620 for switching the terminal device 20C to a "viewing mode", an icon 622 for switching the terminal device 20C to a "solo distribution mode", and an icon 624 for switching the terminal device 20C to a "joint distribution mode" ("collaboration distribution mode"), etc.

[0187] The terminal device 20C can display a pointer (laser pointer) 630 that operates in accordance with an operation by the user A on a tracking device attached or held by the user A, for example, the right hand controller 102 or the left hand controller 104 in this case, in association with the input object 610. The terminal device 20C can draw the pointer 630 as a laser light flowing from the bottom to the top on the paper surface. The terminal device 20C can also draw a trajectory 630a of the pointer 630.

[0188] User A can move the pointer 630 on the surface of the input object 610 by moving the right hand controller 102 or the left hand controller 104. User A can select a mode corresponding to an icon by positioning the pointer 630 on any of the icons 620, 622, or 624 and pressing any of the buttons on the right hand controller 102 or the left hand controller 104. This allows user A to operate a pseudo-displayed terminal device using a tracking device such as a right hand controller (left hand controller) as if he or she were operating the terminal device by touching the touch panel of the terminal device such as a smartphone.

[0189] Terminal device 20C can also display second video 640 in input object 610. Second video 640 can display avatar 450 viewed from the front, arranged in the first virtual space, which moves based on motion data indicating the motion of user A.

[0190] The terminal device 20C can display such an input object 610 movable in conjunction with a part (here, the left hand 450a) of the avatar 450 of the user A included in the first video 600. Specifically, for example, in response to the user A moving the left hand controller 104, the terminal device 20C can move the left hand 450a of the avatar 450 in the first video 600, and move the input object 610 accordingly.

[0191] Furthermore, the terminal device 20C can display such an input object 610 as a pseudo-represented terminal device (pseudo terminal device) as illustrated in Fig. 15. The pseudo terminal device may include, as one aspect, a pseudo-represented pseudo smartphone (pseudo smart phone).

[0192] Furthermore, in one embodiment, the terminal device 20C can display the first video 600 and the input object 610 in different regions on a display unit (such as the HMD 100) that is the output device 26. For example, the terminal device 20C can display the first video 600 in a portion of a specific region included in a display region formed by the display unit, such as the right, left, upper or lower portion, and display the input object 610 in the remaining region of the specific region. In another preferred embodiment, the terminal device 20A can display the input object 610 inside the region in which the first video 600 is displayed, as exemplified in FIG. 15.

[0193] Returning to FIG. 14, next, in ST504, terminal device 20C can determine whether or not any mode (here, for example, "viewing mode", "solo delivery mode", or "shared delivery mode") has been selected by user A while executing the "preparation mode". If terminal device 20C determines that none of the modes has been selected, it can continue to execute ST502. On the other hand, if terminal device 20C determines that any of the modes has been selected, it can proceed to a step corresponding to the selected mode. Specifically, if terminal device 20C determines that the "viewing mode" has been selected, it can proceed to ST506, if it determines that the "shared delivery mode" has been selected, it can proceed to ST508, and if it determines that the "solo delivery mode" has been selected, it can proceed to ST510.

[0194] In ST506, the terminal device 20C can execute a "viewing mode" which is a mode for displaying a video provided by a distributor selected by the user A. The user A can cause the terminal device 20C to execute the "viewing mode" by, for example, aligning the pointer 630 shown in Fig. 15 with the "viewing mode" icon 620 and pressing a button on the right hand controller 102 or the left hand controller 104.

[0195] The terminal device 20C can store mode identification data indicating the "viewing mode" from when it transitions to the "viewing mode" until it ends the "viewing mode." As described above, at any timing, the terminal device 20C can determine which mode is being executed by referring to the mode identification data.

[0196] FIG. 16 is a diagram showing an example of a screen displayed by a terminal device when the "viewing mode" is executed in the communication system shown in FIG. 1. As shown in FIG. 16, the terminal device 20C, which is a VR terminal device, can display a first video (fourth type of video) 700 that operates based on motion data related to the motion of user A when the "viewing mode" is executed. As described above in "5. Platform", the first video 700 is drawn from the viewpoint of user A, and the viewpoint can change based on the motion data. In addition, the first video 700 does not display the avatar 450 of user A.

[0197] Furthermore, the terminal device 20C can display an input object 710 in combination with the first video 700. The terminal device 20C can display the handle name of at least one distributor currently distributing a video in the form of an icon in the input object 710. Here, as an example, the terminal device 20C can display six icons displaying the handle names "Mr. U", "Mr. V", "Mr. W", "Mr. X", "Mr. Y", and "Mr. Z", respectively. The user can determine the distributor to be viewed by placing the pointer on an icon corresponding to a desired one of these distributors and selecting it. Here, user A can select the "Mr. Y" icon to view a video distributed by "Mr. Y".

[0198] Accordingly, the terminal device 20C can display a first video 710 as exemplified in Fig. 17. This first video (fourth type of video) 700 is a video in which an avatar 210 that moves based on the movement of a distributor "Mr. Y" in a first virtual space is arranged. The first video 700 does not display the avatar 450 of the user A.

[0199] Furthermore, the terminal device 20C can display an input object 710 in combination with the first video 700. The input object 710 can display a second video 720 that places an avatar 210 that moves based on the movement of the distributor "Mr. Y" in the first virtual space and displays the avatar 210 as viewed from the front. The second video 720 displayed on the input object 710 when the "viewing mode" is executed differs from the second video 640 displayed on the input object 610 when the above-mentioned "preparation mode" is executed in that the second video 720 does not display the avatar 450 of the user A.

[0200] The second video 720 may be the same as the video commonly displayed on the terminal devices 20 of the viewers. In addition, the second video 720 may display each avatar (here, avatar 210) as viewed from the position of a virtual camera disposed in the first virtual space (three-dimensional space) used in the first video 700.

[0201] Furthermore, the input object 710 may include a number of icons, for example, an icon 730 for switching the terminal device 20C to a "shared distribution mode", an icon 732 for switching the terminal device 20C to a "preparation mode", and the like.

[0202] In this manner, the terminal device 20C can execute the "viewing mode."

[0203] 14, next, in ST512, terminal apparatus 20C can determine whether or not another mode has been selected by user A. If terminal apparatus 20C determines that another mode has not been selected, it can return to ST502 described above and execute the "preparation mode."

[0204] On the other hand, if terminal device 20C determines that another mode has been selected, it proceeds to ST514 and can execute the selected mode. Here, consider a case in which, in ST506 described above, user A selects the "shared distribution mode" icon 730 shown in Fig. 17, and terminal device 20C ends the "viewing mode", passes through ST512, and executes the "shared distribution mode" as the selected mode in ST514.

[0205] In ST514, the terminal device 20C can execute a "joint distribution mode" in which the user A distributes a video in collaboration with at least one distributor. This "joint distribution mode" can be started, for example, by any of the procedures exemplified below. When User A (guest user) sends a request (for example, via the screen shown in FIG. 17) to the distributor (host user) of the video he / she is viewing in “viewing mode” and the distributor approves the request When User A (host user) is streaming a video alone in "solo streaming mode", one of the viewers (guest users) sends a request to User A and User A approves the request. When User A (host user) is streaming a video alone in "solo streaming mode" and sends a request to one of the viewers (guest user) to invite that viewer to become a co-streamer, and that viewer accepts the request.

[0206] For example, the host user and the guest user can jointly distribute a video by performing the operations exemplified below between the host user and the guest user. (i) The host user's terminal device 20 receives request data from the guest user's terminal device 20 via the server device 30 (or the host user's terminal device 20 sends request data to the guest user's terminal device 20 via the server device 30). (ii) The host user's terminal device 20 sends approval data indicating that the request is accepted to the guest user's terminal device 20 via the server device 30 (or the guest user's terminal device 20 sends approval data indicating that the request is accepted to the host user's terminal device 20 via the server device 30). (iii) The server device 30 transmits a list indicating the co-distributors to the terminal device 20 of the host user and the terminal device 20 of the guest user. (iv) The terminal device 20 of the host user transmits the operation data to the server device 30 . (v) The terminal device 20 of the guest user transmits the operation data to the server device 30 . (vi) The server device 30 transmits the action data received in (iv) and (v) above as one set of action data to the terminal device 20 of the host user, the terminal device 20 of the guest user, and the terminal device 20 of each viewer. (vii) The operations thereafter are as described in “5. Platform” above.

[0207] In addition, the terminal device 20C can store mode identification data indicating the "joint distribution mode" from when it transitions to the "joint distribution mode" until it ends the "joint distribution mode." As described above, at any timing, the terminal device 20C can determine which mode it is executing by referring to the mode identification data.

[0208] Note that even while terminal device 20C is running the "joint distribution mode" (user A is distributing a video jointly with another distributor), user A can operate terminal device 20C (or the other distributor can operate terminal device 20) to send a request to one of the viewers to invite that viewer to become a co-distributor. If that viewer operates terminal device 20 to approve the request, user A, the other distributor, and that viewer (new distributor) can jointly distribute a video.

[0209] FIG. 18 is a diagram showing an example of a screen displayed by a terminal device when the "joint distribution mode" is executed in the communication system shown in FIG. 1. As shown in FIG. 18, the terminal device 20C, which is a VR terminal device, can display a first video (third type of video) 800 that operates based on motion data related to the motion of user A. As described above in "5. Platform", the first video 800 is drawn from the viewpoint of user A, and the viewpoint can change based on the motion data. In addition, the first video 800 can display the avatar 210 of the distributor "Mr. Y" and the avatar 450 of user A (here, the left hand 450a of the avatar 450, etc.). The avatar 210 displayed in the first video 800 operates according to the motion data related to the motion of distributor "Mr. Y", and the avatar 450 displayed in the first video 800 operates according to the motion data indicating the motion of user A.

[0210] Furthermore, the terminal device 20C can display an input object 810 in combination with the first video 800. The input object 810 can include a plurality of icons, for example, an icon 820 for transitioning the terminal device 20C to a "preparation mode" and an icon 822 for transitioning the terminal device 20C to a "viewing mode".

[0211] Terminal device 20C can also display second video 830 in input object 810. Second video 830 can display avatar 210, which moves based on motion data showing the motion of distributor "Mr. Y," and avatar 450, which moves based on motion data showing the motion of user A, in the first virtual space, with avatar 210 and avatar 450 viewed from the front.

[0212] The second video 830 may be the same as the video commonly displayed on the terminal devices 20 of the viewers. In addition, the second video 830 may display each avatar (here, avatar 210 and avatar 450) as viewed from the position of a virtual camera arranged in the first virtual space (three-dimensional space) used in the first video 800.

[0213] In this manner, the terminal device 20C can execute the "joint distribution mode."

[0214] 14, in ST514, terminal device 20C can execute and end the selected mode, "joint distribution mode." After this, terminal device 20C returns to ST512 described above.

[0215] It has been explained that if the "joint distribution mode" is selected in ST504, terminal device 20C executes the "joint distribution mode" in ST508, and if the "solo distribution mode" is selected in ST504, terminal device 20C executes the "solo distribution mode" in ST510. The process executed by terminal device 20C in relation to the "joint distribution mode" in ST508 is as described above in relation to ST514.

[0216] The process executed by terminal device 20C in ST510 in relation to the "single delivery mode" will now be described.

[0217] The terminal device 20C can store mode identification data indicating the "single delivery mode" from when it transitions to the "single delivery mode" until it ends the "single delivery mode". As described above, at any timing, the terminal device 20C can determine which mode it is executing by referring to the mode identification data.

[0218] FIG. 19 is a diagram showing an example of a screen displayed by a terminal device when the "single distribution mode" is executed in the communication system shown in FIG. 1. As shown in FIG. 19, the terminal device 20C, which is a VR terminal device, can display a first video (second type of video) 900 that operates based on motion data related to the motion of user A. As described above in "5. Platform", the first video 900 is drawn from the viewpoint of user A, and the viewpoint can change based on the motion data. The first video 900 can also display an avatar 450 of user A (here, the left hand 450a of the avatar 450, etc.). The avatar 450 displayed in the first video 900 operates according to the motion data indicating the motion of user A.

[0219] Furthermore, the terminal device 20C can display an input object 910 in combination with the first video 900. The input object 910 can include a plurality of icons, for example, an icon 920 for transitioning the terminal device 20C to a "preparation mode", an icon 922 for transitioning the terminal device 20C to a "viewing mode", an icon 924 for transitioning the terminal device 20C to a "shared distribution mode", and the like.

[0220] Terminal device 20C can also display second moving image 930 in input object 910. Second moving image 930 can display avatar 450 viewed from the front, by arranging avatar 450 that moves based on motion data indicating a motion of user A in the first virtual space.

[0221] The second video 930 may be the same as the video commonly displayed on the terminal devices 20 of the viewers. The second video 930 may display each avatar (here, avatar 450) as viewed from the position of a virtual camera arranged in the first virtual space (three-dimensional space) used in the first video 900.

[0222] In this manner, the terminal device 20C can execute the "single delivery mode".

[0223] As described above, the terminal device 20C can display any one of the four types of moving images selected in accordance with an input to an input object by the user A. The four types of moving images are summarized as follows. A first type of video: a video that is displayed when the terminal device 20C determines that the user A is in a preparation mode in which the user A is neither a distributor nor a viewer, and in which an avatar of the user A that moves based on the movement of the user A in the first virtual space is arranged. A second type of video: a video that is displayed when the terminal device 20C determines that the terminal device 20C is executing a single distribution mode in which the user A is a distributor, and in which an avatar of the user A that moves based on the movement of the user A in the first virtual space is arranged. A third type of video: a video displayed when the terminal device 20C determines that the terminal device 20C is executing a joint distribution mode in which the user A is a joint distributor, and in which an avatar of the user A that moves based on the movement of the user A and an avatar of at least one other distributor that moves based on the movement of the at least one other distributor are arranged in the first virtual space. A fourth type of video: a video that is displayed when the terminal device 20C determines that a viewing mode in which the user A is a viewer is being executed, and in which an avatar of at least one distributor that moves based on the movement of the at least one distributor is arranged in the first virtual space.

[0224] 6-2.Modifications As described above, the terminal device 20C can display the second video in the input object while executing any one of the "preparation mode", "viewing mode", "joint distribution mode" and "solo distribution mode". When the terminal device 20C is executing a mode other than the "viewing mode" (i.e., the "preparation mode", "joint distribution mode" and "solo distribution mode"), the second video can display an avatar of the user A that moves based on the movement of the user A of the terminal device 20C. The avatar of the user A displayed in this second video is also commonly displayed on the terminal devices (including the VR terminal device and the smartphone terminal device) of each viewer and each distributor (in the case of the "joint distribution mode" and the "solo distribution mode"). Therefore, it is considered that the user A wants to check how his / her avatar appears to others (appearance or behavior, etc.) by looking at the avatar of the user A displayed in this second video.

[0225] In view of such circumstances, the terminal device 20C can display the input object (and thus the second video) in a first size when executing the "viewing mode", and can display the input object (and thus the second video) in a second size larger than the first size while executing the "joint distribution mode" or the "solo distribution mode". This allows the user A to check the appearance of his / her avatar displayed in the second video at any timing while distributing a video as a distributor, more easily and effectively. In addition, it can be said that the situation in which the user A carefully checks the appearance of his / her avatar over a certain amount of time is while the terminal device 20C is executing the "preparation mode". Therefore, the terminal device 20C can display the input object (and thus the second video) in a second size larger than the first size even while executing the "preparation mode".

[0226] Also, in the above-mentioned "viewing mode", the terminal device 20C of the user A can display messages and / or comments, etc., sent from any of the viewers to the distributor in the second video displayed in the input object. Similarly, in the "solo distribution mode" or "joint distribution mode", the terminal device 20C of the user A can display messages and / or comments, etc., sent from any of the viewers to the distributor, user A, or other distributors, in the second video displayed in the input object. In such a case, when a number of messages, etc., exceeding the display area of ​​the input object are sent from the viewers, it is considered that the oldest messages are sequentially erased from the input object and new messages are displayed. In contrast, typically, the display area of ​​the first video may be larger than the display area of ​​the input object. Therefore, the terminal device 20C can display messages, etc., sent from each viewer to the distributor in the first video. In a preferred embodiment, the terminal device 20C can display the above-mentioned messages, etc., in an area of ​​the display area of ​​the first video that corresponds to the background located behind the input object (for example, area 845, etc., illustrated in FIG. 18).

[0227] Furthermore, in the above-mentioned "viewing mode," the terminal device 20C of user A can send a gift (token) to the distributor by operating an icon or the like displayed in the input object. In this case, the terminal device 20C can more easily send a gift to the distributor by using the first moving image. A specific example will be described with reference to FIG. 20.

[0228] Fig. 20 is a diagram showing another example of a screen displayed by a terminal device when the "viewing mode" is executed in the communication system shown in Fig. 1. As exemplified in Fig. 20, the terminal device 20C can display a plurality of objects (three objects in Fig. 20) that are candidates for transmission to the side, above, or below the distributor's avatar 210 in the first video 700. In addition, the terminal device 20C can sequentially switch the plurality of objects displayed in the first video 700 to another object in response to the user A performing a predetermined gesture (e.g., a gesture of shaking the hand controller to the right or left) using the right hand controller 102 (or the left hand controller 104).

[0229] For example, the user A can determine and transmit the object to be transmitted by aligning the pointer 630 with a desired object among the multiple objects displayed in this manner, pressing any button on the right hand controller 102 (or the left hand controller 104), and making a gesture of throwing the hand controller being held. Here, it is assumed that the user A determines the object to be transmitted as a "bow tie". In response to this, the broadcaster "Mr. Y" can operate the terminal device 20 to make the avatar 210 wear the "bow tie" received as a gift. Alternatively, the terminal device 20 of the broadcaster "Mr. Y" can automatically make the avatar 210 wear the "bow tie" received as a gift without the intervention of the broadcaster "Mr. Y". As a result, the terminal device 20C can display the avatar 210 wearing the bow tie in the first video 700 and / or the second video 720. In this way, the terminal device 20C can provide a user interface that allows the user to determine and send a gift simply and effectively by using the larger first moving image rather than using a smaller input object.

[0230] In addition, in the above-mentioned various examples, the description has been given with a focus on a method (client rendering method) in which the terminal device 20C executes the operations in each step (ST502 to ST514). However, in at least one target step among ST502 to ST514, it is also possible to adopt a method (server rendering method) in which the operations related to the generation of moving images that have been described as being executed by the terminal device 20C are executed by one server device 30 alone or by multiple server devices 30 in a shared manner. In this case, the one or more server devices 30 execute various operations related to the at least one target step instead of the terminal device 20C, thereby generating a first moving image and a second moving image (input object) and transmitting them to the terminal device 20C. As described in "5. Platform" above, the server device 30 receives motion data from the terminal device 20 of each distributor, and can execute the generation of various moving images using the motion data instead of each terminal device 20. Meanwhile, the server device 30 does not necessarily receive motion data from the terminal device 20 of each viewer. Therefore, when the server rendering method is adopted, the server device 30 may receive the motion data from the terminal device of each viewer for each unit time or at any timing.

[0231] Also, in the various examples described above, the explanation has been focused on a method (client rendering method) in which each terminal device 20C executes operations related to the generation of the first moving image and the second moving image (input object), etc. However, the terminal device 20C can also adopt a method (browser rendering method) in which the terminal device 20C receives a web page from the server device 30 or the web server device 30 via a web browser and executes a computer program included in the web page to perform operations related to the generation of the first moving image and the second moving image (input object), etc.

[0232] Furthermore, in the various examples described above, the second video (input object) may display, for example, avatar 450 of user A, which moves based on motion data related to the motion of user A. Here, while terminal device 20C is executing any one of the "preparation mode", "joint distribution mode", and "solo distribution mode", it may invert (mirror) the image of avatar 450 seen from a virtual camera arranged in the first virtual space (i.e., the appearance of avatar 450) and display it in the second video (input object).

[0233] In this case, the avatar 450 arranged in the first virtual space moves based on the motion data related to the motion of the user A, and therefore can move according to the motion of the user A (generally faithfully in comparison with "inversion"). In contrast, the avatar 450 displayed in the second video (input object) is an inverted (mirrored) image of the avatar 450 seen from a virtual camera arranged in the first virtual space. Therefore, the appearance of the avatar 450 displayed in the second video (input object) corresponds to an inverted (mirrored) appearance of the avatar 450 in the first virtual space, which acts as an alter ego of the user A. From the viewpoint of the user A, the motion of the avatar 450 displayed in the second video (input object) corresponds to the motion of the user A himself / herself as if it were reflected in a mirror. This allows the user A to observe the second video (input object) as if he / she were watching the motion of the avatar 450, which is his / her alter ego, in a mirror placed in front of the user A.

[0234] Note that the second video (input object) displays an inverted image of the avatar 450 as seen from a virtual camera placed in the first virtual space, and icons, messages, comments and / or pointers, etc. displayed in the second video (input object) are not inverted and may be displayed as described above.

[0235] As described above, in various embodiments, the terminal device used by the user can display an input object that displays a second moving image in combination with the first moving image. This allows the user to receive services related to the delivery and / or display of moving images by operating the input object in the same way as when operating a display device (such as a touch panel) of a conventional terminal device, even when using a tracking device worn or held on the body. When the operation method for the input object (such as a user interface or icon display) is substantially the same as or similar to the operation method used in a conventional terminal device, a user who is accustomed to the operation method for the conventional terminal device can easily use the operation method for the input object without spending a lot of time.

[0236] Furthermore, the terminal device can change the input object that displays the second moving image according to the type of the first moving image that is displayed in combination with the input object. For example, the terminal device can enlarge or reduce the size of the input object that displays the second moving image according to the type of the first moving image that is displayed.

[0237] As a result, when a user distributes a video as a distributor or when preparing to distribute a video, the user can check the appearance of his / her own avatar more effectively by viewing the enlarged second video included in the input object. By displaying the second video in an enlarged manner in this manner, the user can more easily check his / her own avatar, which is enlarged and easy to see. Furthermore, when a user, as a viewer, watches a video distributed by a distributor, the user can more effectively enjoy the distributor's avatar displayed in the first video by using the first video, which is displayed in a relatively wide area as the input object is reduced.

[0238] Furthermore, the terminal device automatically changes the size of the input object depending on the state of the user, i.e., whether the user is distributing a video (or preparing to distribute a video) or whether the user is watching a video, thereby eliminating the need for time-consuming and laborious operations by the user via a touch panel or user interface, etc. This not only improves usability, but also reduces power consumption of the terminal device and / or tracking device.

[0239] Furthermore, since the user does not need to hold a physically existing smartphone or tablet, but only needs to operate an input object virtually represented on a display (which is intuitively easy to accept, like a mirror), the user can freely move his / her hands and feet when distributing a video or preparing to distribute a video (or when watching a video). Therefore, in this sense as well, usability can be improved.

[0240] 7. About Closet Mode 7-1. Basic configuration Next, a mode (closet mode) for changing the display method of the user's avatar in combination with the above-mentioned first moving image will be described with reference to Fig. 21. Fig. 21 is a flow diagram illustrating another example of the operation executed by the communication system 1 shown in Fig. 1.

[0241] Again, for convenience, the operations performed in the communication system 1 will be described in relation to the terminal device (VR terminal device) 20C of the above-mentioned user A, but such operations are similarly applicable to the terminal device (VR device) 20 of any other user.

[0242] Referring to FIG. 21, first, in ST1000, the terminal device 20C can display a first moving image. As described above, the first moving image may be a moving image that operates based on the motion data related to the user A and depicts a first virtual space (three-dimensional virtual space) as seen from the viewpoint of the user A. For example, the first moving image may be a first moving image that is displayed when the terminal device 20C is executing the above-mentioned "preparation mode", "joint distribution mode", or "solo distribution mode". Details of this first moving image are as described above, and therefore will not be described again.

[0243] Next, in ST1002, terminal device 20C can activate the closet mode by user A performing a specific operation indicating that closet mode is to be executed. Note that terminal device 20C can execute the closet mode in any situation where user A wishes to objectively observe his / her avatar. This situation may include at least one of the following examples. - A situation where User A attaches or detaches objects, gifts, and / or items to or from his / her avatar - A situation where User A checks his / her avatar as a preparation step before distributing a video that includes his / her avatar - A situation in which User A is live streaming a video that includes his / her avatar Furthermore, the specific operation referred to here may be, for example, at least one of the operations exemplified below. User A holds the right hand controller 102 (or the left hand controller 104) and performs a specific gesture. User A presses any button on the right hand controller 102 (or the left hand controller 104). User A wears the HMD 100 on his / her head and performs a specific gesture (for example, shaking his / her head from side to side). User A selects a specific icon displayed in the input object described above, etc.

[0244] By activating the closet mode, terminal device 20C can display the isolated video in ST1004 in place of the first video (for example, first video 400A shown in FIG. 13) that has been displayed up until that point. The isolated video (described as "second video" for convenience in the claims) may be a video that moves based on motion data related to the motion of user A and depicts the first virtual space including user A's avatar as viewed from a remote viewpoint at a distance from user A's avatar.

[0245] Fig. 22 is a diagram showing an example of a screen displayed when the "closet mode" is executed by the terminal device in the communication system shown in Fig. 1. As exemplified in Fig. 22, the isolated video 1100 may be a video depicting the first virtual space (three-dimensional virtual space) including the avatar 450 of the user A as viewed from a isolated viewpoint at a distance from the avatar 450.

[0246] The first video (here, the first video 400A shown in FIG. 13) that the terminal device 20C displayed before displaying this isolated video 1100 is drawn from the viewpoint of the avatar 450 (user A who is in approximately the same position as the avatar 450), and therefore only a part of the avatar 450 (for example, the left hand 450a) is displayed. In contrast, the isolated video 1100 is drawn from a distant viewpoint from the avatar 450, and therefore a large part of the avatar 450 (in FIG. 22, the whole body seen from behind) may be displayed. Note that FIG. 22 shows the first virtual space drawn from a distant viewpoint located at the upper left rear of the avatar 450, but it is possible to place the isolated viewpoint at any position distant from the avatar 450. For example, the terminal device 20C can display the isolated video 1100 in which the avatar 450 (the face, etc.) is drawn from the front by placing the isolated viewpoint in front of the avatar 450.

[0247] Furthermore, at the time when the terminal device 20C starts displaying the quarantine video 1100, the terminal device 20C can draw the avatar 450 in the quarantine video 1100 so as to maintain the pose that the avatar 450 expressed in the first video that had been displayed up until that point. Alternatively, at the time when the terminal device 20C starts displaying the quarantine video 1100, the terminal device 20C can draw the avatar 450 in the quarantine video 1100 so as to reflect any one pose automatically selected from at least one default pose prepared in advance or any one pose selected by the user A.

[0248] Also, for the sake of simplicity, the isolated video 1100 shown in Fig. 22 does not display the avatar 220 and the avatar 210 shown in Fig. 13. However, if these avatars are visible from the isolated viewpoint, the isolated video 110 can also depict these avatars. This also applies to Figs. 22, 25, and 27, which will be referred to later.

[0249] Here, the isolated viewpoint will be explained in comparison with the viewpoint of user A (avatar 450). Fig. 23 is a diagram conceptually showing the viewpoint of a user (avatar) used in generating the first moving image in the communication network 1 shown in Fig. 1. Fig. 24 is a diagram conceptually showing the isolated viewpoint used in generating the isolated moving image in the communication network shown in Fig. 1.

[0250] 23 related to the first moving image, head 460 of user A is shown as viewed from above, and head 470 of avatar 450 is shown as viewed from above. As shown in FIG. 23, the first moving image depicts the first virtual space as viewed from the viewpoint of user A or avatar 450 in a state in which head 470 of avatar 450 is placed on top of head 460 of user A. Therefore, from such a viewpoint, the first moving image cannot depict the entire body of avatar 450, and depicts only part 450a of avatar 450, as shown in FIG.

[0251] 24 related to the isolation video, head 460 of user A is also shown as viewed from above, and head 470 of avatar 450 is also shown as viewed from above. As shown in Fig. 24, the isolation video renders the first virtual space by placing a viewpoint (isolation viewpoint) on head 460 of user A that is isolated, for example, at a distance L from head 470 of avatar 450. Therefore, with such an isolation viewpoint, the isolation video can render a larger portion of avatar 450.

[0252] Returning to FIG. 21, next, in ST1006, the terminal device 20C can control the movement of the avatar 450 displayed in the separation video 1100. The avatar 450 displayed in the first video 400A (FIG. 13, etc.) can move based on the movement data related to the movement of the user A, as described above. The avatar 450 displayed in the isolation video 1100 (FIG. 22, etc.) can also basically move based on the movement data related to the movement of the user A. That is, the avatar 450 displayed in the isolation video 1100 basically moves reflecting the movement of the user A. For example, if the user A makes a gesture of raising the right hand holding the right hand controller 102, in response to this, the avatar 450 can also raise its right hand in the isolation video 1100 exemplified in FIG. 22.

[0253] In one embodiment, the terminal device 20C can control the movement of the avatar 450 so as not to change the position (coordinates) of the avatar 450 in the isolation video 1100, regardless of the content of the movement data of the user A. For example, even if the user A presses any button on the right hand controller 102 (or the left hand controller 104) prepared to generate movement data indicating the forward or backward movement of the avatar 450, the terminal device 20C can invalidate such movement data indicating the forward or backward movement. Furthermore, for example, when the user A moves the head wearing the HMD 100 forward or backward, the position of the HMD 100 moves, and the terminal device 20C can invalidate the movement data indicating the forward or backward movement generated by the HMD 100.

[0254] The closet mode may be executed for the purpose of user A checking the appearance of avatar 450. It is useful to prevent avatar 450 from moving to a position in the first virtual space that is not intended by user A due to a movement of user A while user A is gazing at avatar 450 to check the appearance of avatar 450. Therefore, terminal device 20C can control the movement of avatar 450 so as not to change the position (coordinates) of avatar 450, regardless of the movement data of user A.

[0255] Furthermore, in one embodiment, the terminal device 20C can control the movement of the avatar 450 so as not to rotate the avatar 450 around a rotation axis extending in the vertical direction in the first virtual space in the isolation video 1100, regardless of the contents of the movement data of the user A. Here, in FIG. 22, for ease of explanation, a rotation axis O extending in the vertical direction in the first virtual space is shown, but the isolation video 1100 does not need to display such a rotation axis O (the rotation axis O may be displayed).

[0256] The process of preventing avatar 450 from rotating around rotation axis O will be described with reference to Fig. 24 used above. In Fig. 24(a), head 460 of user A is positioned at a distance L from head 470 of avatar 450. The viewpoint seen from head 460 of user A thus positioned becomes the isolated viewpoint.

[0257] In the state shown in FIG. 24(a), in order to change the observation part in the avatar 450 while maintaining the distance L from the head 460 of the user A, for example, the user A executes an action of pressing any button provided on the right hand controller 102 (or the left controller 104), thereby rotating the viewpoint of the user A 90 degrees counterclockwise on the paper. The head 460 of the user A can be moved to the position shown in FIG. 24(b) by the action data based on such an action of the user A. However, according to such action data, the avatar 450 also rotates 90 degrees counterclockwise in the same manner. As a result, as is clear from FIG. 24(b), the head 460 of the user A continues to face the back of the avatar 450 regardless of whether the head 460 is in the state shown in FIG. 24(a) or FIG. 24(b). In this way, when the avatar 450 is rotated in conjunction with the action data indicating the action of the user A, the user A cannot change the observation part in the avatar 450.

[0258] On the other hand, in the case where avatar 450 is not rotated around rotation axis O extending in the vertical direction of avatar 450, when user A operates right hand controller 102 or the like to rotate user A's viewpoint 90 degrees counterclockwise on the page in the state shown in Fig. 24(a), user A's head 460 can face the right side of avatar 450, as shown in Fig. 24(c). Therefore, by not rotating avatar 450 around rotation axis O, user A can change the observed part of avatar 450.

[0259] In this way, by terminal device 20C controlling the movement of avatar 450 in isolation video 1100 so as not to rotate avatar 450 around a rotation axis extending vertically in the first virtual space, regardless of the content of user A's movement data, user A can objectively observe the appearance of avatar 450 (with at least one object attached or detached) from a desired angle.

[0260] Note that preventing avatar 450 from rotating around rotation axis O means that even if user A wearing HMD 100 turns his / her head to the right or left (rotates his / her head clockwise or counterclockwise around a rotation axis extending vertically), the head of avatar 450 cannot turn to the right or left (i.e., cannot swing its head) (the direction / angle in which avatar 450's head faces does not change).

[0261] Therefore, in one embodiment, terminal device 20C may also have a function that enables rotation of avatar 450. Specifically, returning to Fig. 21, in ST1008, terminal device 20C may display a user interface for rotating the avatar.

[0262] For example, the terminal device 20C can display a specific user interface 1200 in the isolation video 1100 as shown in FIG. 22. This user interface 1200 can include, for example, a right icon 1202 for rotating the avatar 450 clockwise around the rotation axis O, and a left icon 1204 for rotating the avatar 450 counterclockwise around the rotation axis O. Furthermore, the terminal device 20C can display the above-mentioned pointer 630 in association with the user interface 1200. For example, the user A moves the right hand controller 102 or the like to align the pointer 630 with the right icon 1202 (or the left icon 1204) and presses any button provided on the hand controller, whereby the terminal device 20C can rotate the avatar 450 clockwise (or counterclockwise) around the rotation axis O at an angle corresponding to the number of times or time of the press. An example is shown in FIG. 25.

[0263] Similarly, although not shown in FIG. 22, the terminal device 20C can display an interface for rotating not only the rotation axis O of the entire avatar 450 but also each of a plurality of elements including a plurality of parts and a plurality of bones included in the avatar 450 around the rotation axis in the isolation video 1100. The above-mentioned plurality of elements may include, without being limited to, the head, the right arm, the left arm, the right leg, the left leg, the upper body, and / or the lower body. When the user A selects an object (an icon for rotating the element clockwise, an icon for rotating the element counterclockwise, etc.) associated with a desired element in the user interface displayed in this way, the terminal device 20C can rotate and move the element in the avatar 450 in the isolation video 1100, that is, fine-tune the pose expressed by the avatar 450. Here, rotating and moving the above elements may include, but is not limited to, tilting the head of the avatar 450 diagonally, rotating only the head of the avatar 450, stretching only the right hand of the avatar 450 horizontally, kicking up only the left foot of the avatar 450, etc. This allows the terminal device 20C to cause the avatar 450 to perform a wide variety of actions in the isolated video 1100 while reducing the amount of motion data received from the HMD 100, the right hand controller 102, and / or the left hand controller 104. This also makes it possible to reduce power consumption of the terminal device 20C.

[0264] Alternatively or in addition to this, the terminal device 20C may display a specific user interface including an indication such as "If you swing the right hand controller to the right, you can rotate the avatar to the right" or "If you swing the left hand controller to the left, you can rotate the avatar to the left" in the isolated video 1100, although this is not shown in Fig. 22 and Fig. 25. In this case, the user A operates the right hand controller and / or the left controller as instructed, whereby the terminal device 20C can rotate the avatar 450 clockwise or counterclockwise around the rotation axis O (and / or operate a desired element in the avatar 450).

[0265] Alternatively or in addition to this, the terminal device 20C may display a specific user interface including a display related to a temporary release mode that temporarily permits rotation of the avatar 450 (i.e., rotates the avatar 450 in the same manner as in the first video according to the motion data related to the motion of the user A) in the isolated video 1100, although this is not shown in Fig. 22 and Fig. 25. In this case, the terminal device 20C can execute the temporary release mode by the user A pressing any button provided on the right hand controller or the left controller, or by operating the pointer 630 using the right hand controller or the left controller, etc.

[0266] Returning to FIG. 21, next, in ST1010, the terminal device 20C can change the isolated viewpoint applied to the isolated video. Specifically, for example, the terminal device 20C can display a specific user interface for changing the isolated viewpoint in the isolated video 1100, although this is not shown in FIG. 22 and FIG. 25. This user interface can include at least one icon or the like. For example, the user A can move the right hand controller 102 or the like to align the pointer 630 (see FIG. 22, etc.) with such an icon and press any button provided on the hand controller. This allows the terminal device 20C to change the distance between the head 470 of the avatar 450 and the head 460 of the user A (user A's viewpoint) and / or the coordinates (x, y, z) of the head 460 of the user A (user A's viewpoint) in the first virtual space, as shown in FIG. 24. The terminal device 20C can generate and display an isolated video depicting the first virtual space seen from the isolated viewpoint changed in this way.

[0267] Alternatively or in addition to this, the terminal device 20C may display a specific user interface including an indication such as "Pressing the button on the right hand controller increases the distance between the avatar and the user's viewpoint", "Pressing the button on the left hand controller decreases the distance between the avatar and the user's viewpoint", "Pushing the joystick on the right hand controller changes the user's viewpoint in the horizontal direction", and "Pushing the joystick on the left hand controller changes the user's viewpoint in the vertical direction" in the isolation video 1100, although this is not shown in Fig. 22 and Fig. 25. In this case, the user A operates the right hand controller and / or the left controller as instructed, so that the terminal device 20C can change the distance between the head 470 of the avatar 450 and the head 460 of the user A (user A's viewpoint) and / or the coordinates (x, y, z) of the head 460 of the user A (user A's viewpoint) in the first virtual space, as shown in Fig. 24.

[0268] In another example, the terminal device 20C can use the viewpoint of an avatar of another distributor present in the first virtual space as the isolated viewpoint. For example, in a case where the terminal device 20C has been displaying the first video 400A shown in FIG. 13 until the closet mode is activated in ST1002, the user A is currently distributing a video in collaboration with the first distributor corresponding to the avatar 210 and the second distributor corresponding to the avatar 220. Note that the avatar 210 moves based on the actions of the first distributor using a plurality of tracking devices as described above.

[0269] In this case, the terminal device 20C can display the first video 400A in which the avatar 210 is placed at the position illustrated in FIG. 13 because the terminal device 20C continuously recognizes the coordinates of the avatar 210 in the first virtual space based on the operation data received from the terminal device 20 of the first distributor via the server device 30. Therefore, the terminal device 20C can use the coordinates of the avatar 210, which are continuously recognized in this manner, as the coordinates at which the isolated viewpoint should be placed. As a result, the terminal device 20C can display the isolated video 1100, for example, as shown in FIG. 26, by using the viewpoint of the avatar 210 as the isolated viewpoint. As an example, the isolated video 1100 can display the avatar 450 of the user A from the front. In the example illustrated in FIG. 26, the avatar 220 of another distributor is in the field of view as seen from the isolated viewpoint, so that the isolated video 1100 can depict not only the avatar 450 but also the avatar 220.

[0270] It should be noted that this isolated video 1100 is rendered from the perspective of the first broadcaster's avatar 210, but is displayed on user A's terminal device 20C and may not be displayed on the first broadcaster's terminal device 20.

[0271] In another example, the terminal device 20C can use the viewpoint of an avatar of one of the viewers present in the first virtual space as the isolated viewpoint. According to the principle described in "5. Platform" above, the terminal device 20C of user A, who is a distributor, transmits motion data indicating the motion of this user A to the server device 30. The terminal device (VR terminal device) 20 of the other distributor or the terminal device (VR terminal device) 20 of the viewer can place the avatar 450 of user A, which moves based on the motion data received from the server device 30, in the first virtual space, and generate and display the first moving image drawn from the viewpoint of the other distributor or the viewer.

[0272] The terminal device 20C of user A can also use the viewpoint of the other broadcaster or the viewer in the first virtual space as a remote viewpoint. In this case, the terminal device 20C receives motion data related to the motion of the other broadcaster from the terminal device 20 of the other broadcaster via the server device 30, and can recognize the coordinates of the avatar of the broadcaster in the first virtual space.

[0273] On the other hand, terminal device 20C does not receive any action related to the action of the viewer from terminal device 20 of the viewer via server device 30, and therefore cannot recognize the coordinates of the avatar of the viewer in the first virtual space as it is. Therefore, in order to recognize the coordinates of the avatar of the viewer in the first virtual space, terminal device 20C can receive, from the terminal device of the viewer via server device 30, the coordinates of the avatar of the viewer or action data related to the action of the viewer for calculating the coordinates.

[0274] Even in such a case, the isolated video 1100 can be displayed on the terminal device 20C of the user A, and not on the terminal device 20 of the distributor or the terminal device 20 of the viewer.

[0275] Also, here, a case has been described in which the terminal device 20C uses the viewpoint of an avatar of a distributor and / or a viewer arranged in the first virtual space as the isolated viewpoint used when rendering the isolated video 1100. However, this is merely an example, and the terminal device 20C can also use the viewpoint of any other avatar arranged (present) in the first virtual space as the isolated viewpoint used when rendering the isolated video 1100.

[0276] Returning to FIG. 21, next, in ST1012, terminal apparatus 20C can display a user interface for dressing up avatar 450.

[0277] Fig. 27 is a diagram showing another example of a screen displayed when the "closet mode" is executed by the terminal device in the communication system shown in Fig. 1. As exemplified in Fig. 27, the terminal device 20C can display a plurality of objects (three objects in Fig. 27) that are candidates for attachment to the avatar 450 on the side, above, or below the avatar 450 of the user A in the quarantine video 1100. Note that these multiple objects may include an object sent as a gift from the viewer to the user A.

[0278] The plurality of objects may include, but is not limited to, at least one of the following plurality of objects: - Objects purchased by User A from the company that operates the video distribution service - Object given to User A as a reward from the company User A acquires an object from the company by using any points he or she owns. - Objects that User A has received or purchased from other users - Objects created by User A

[0279] Furthermore, the terminal device 20C can sequentially switch between multiple objects displayed in the isolated video 1100 and other objects in response to user A performing a predetermined gesture (such as a gesture of waving the hand controller to the right or left) using the right hand controller 102 (or the left hand controller 104).

[0280] For example, the user A can align the pointer 630 (pointing object) with a desired object among the multiple objects displayed in this way and press any button on the right hand controller 102 (or the left hand controller 104) to attach the desired object (here, a "hat") to the pointer 630 in the isolation video 1100 (determine this object as an additional object). In this state, the user A can move (so-called drag) the "hat" object attached to the pointer 630 by moving the right hand controller 102 (or the left hand controller 104) until it comes into contact with the avatar 450. After this, the user A can release the "hat" object from the pointer 630 and attach it to the avatar 450 in the isolation video 1100 by releasing any of the buttons on the right hand controller 102 (or the left hand controller 104).

[0281] User A can also detach any object attached to avatar 450 from avatar 450 by performing a similar drag operation using pointer 630 on the object.

[0282] In this manner, avatar 450 with any object attached or detached is displayed in isolation video 1100 (first video and / or input object after closet mode is terminated).

[0283] 21, in ST1014, terminal device 20C can determine whether to continue the closet mode. For example, terminal device 20C can make such a determination based on whether user A has performed an action or operation to end the closet mode.

[0284] If terminal device 20C determines to continue the closet mode, it returns to the above-mentioned ST 1004. On the other hand, if terminal device 20C determines to end the closet mode, it returns to the above-mentioned ST 1000.

[0285] For the sake of convenience, the operations performed in each of ST1004 to ST1014 have been described in this order. However, these steps are not necessarily performed in this order, and may be performed in a different order. Also, a plurality of these steps may be performed in parallel.

[0286] 7-2.Modifications In the various examples described above, a case has been described in which the terminal device 20C switches between a mode (first mode) in which the first video is displayed in a certain area on the display unit and a mode (second mode: closet mode) in which the quarantine video is displayed in the certain area, that is, the terminal device 20C does not display the quarantine video (or the first video) in the certain area during a period in which the first video (or the quarantine video) is displayed in the certain area. In this case, referring to the flow chart shown in FIG. 21, the terminal device 20C can execute a process of generating a first video in parallel so as to be able to display the first video in place of the second video at any timing during a period in which ST1002 to ST1014 are being executed.

[0287] On the other hand, in another example, the terminal device 20C can display the first moving image in a part of the area of ​​the display unit and the quarantine moving image in another part of the area of ​​the display unit in parallel, that is, the first moving image and the quarantine moving image can be displayed in parallel in separate areas (note that the part of the area and the other part of the area of ​​the display unit can partially overlap). In this case, referring to the flow chart shown in FIG. 21, the terminal device 20C can generate and display the first moving image in parallel with steps ST1002 to ST1014 during the period in which these steps are being executed. Note that in this other example, the terminal device 20C can display only one of the first moving image and the quarantine moving image at a certain timing, and display both the first moving image and the quarantine moving image at another timing.

[0288] In this other example, for example, in the first video, terminal device 20C can move avatar 450 of user A based on first action data related to the action of user A, that is, can reflect the action of user A in avatar 450. In this case, terminal device 20C can also display a virtual mirror arranged in the first virtual space and display avatar 450 reflected in this mirror in the first video. This virtual mirror can display avatar 450 drawn as seen from a certain viewpoint (mirror position) arranged in the first virtual space in a mirror-like inversion.

[0289] Furthermore, in this other example, the terminal device 20C can change the viewpoint of the avatar 450 of the user A and / or move the avatar 450 of the user A based on the first action data in the first moving image.

[0290] In addition, in the other example, the terminal device 20C can change the isolated viewpoint in the isolated video based on the first action data, or can keep the isolated viewpoint unchanged. Furthermore, in the isolated video, the terminal device 20C can (i) move the avatar 450 of the user A based on the first action data, or (ii) move the avatar 450 of the user A regardless of the first action data.

[0291] Regarding (ii) above, for example, the terminal device 20C can display the avatar 450 in a stationary state in the isolation video, or can automatically move the avatar 450 regardless of the first motion data.

[0292] Regarding the above (i), for example, terminal device 20C may adopt a configuration in which, in the isolation video, avatar 450 of user A is moved (e.g., arms and legs of avatar 450 are moved) based on a portion of the first motion data, and avatar 450 of user A is not moved (e.g., the position of avatar 450 is not changed) based on the remaining motion data. That is, terminal device 20C may adopt a configuration in which only a portion of the first motion data is used as tracking data, and the remaining motion data is not used as tracking data.

[0293] In this way, examples in which terminal device 20C causes avatar 450 to move based on some of the first action data in the isolation video, and does not cause avatar 450 to move based on the remaining action data, may include, but are not limited to, at least the examples listed below. Move the right and / or left hands of avatar 450, but do not move the right and / or left feet of avatar 450. Only the head of the avatar 450 is moved, and other parts of the avatar 450 are not moved. Other than that, some parts (one or more parts) of the avatar 450 are moved, and other parts of the avatar 450 are not moved.

[0294] Furthermore, the terminal device 20C can also execute an action that combines the above (i) and (ii). Specifically, the terminal device 20C can move some parts (one or more parts) of the avatar 450 based on some of the action data of the first action data, and automatically move other parts (e.g., the remaining parts). For example, the terminal device 20C can move the head of the avatar 450 (or the head and both hands of the avatar 450) based on the action data, and automatically move other parts of the avatar 450 (e.g., the remaining parts).

[0295] In addition, in this other example, the terminal device 20C can also reflect, in the isolation video, any one of a plurality of poses prepared in advance that is selected by the user A or any one of a plurality of poses that is automatically selected, in the avatar 450 of the user A. The plurality of poses can include, for example, but are not limited to, a cutesy pose, a pose expressing a peace sign with the right hand, a pose drawing a heart with both hands, and / or a pose raising both hands upward.

[0296] In this way, by not using at least a portion of the first action data in the isolated video for drawing avatar 450 (stopping some tracking), terminal device 20C can objectively realize customization of user A's avatar 450 while reducing the load required for the drawing process (and thus reducing the power consumption of terminal device 20C).

[0297] In the various examples described above, the terminal device 20C can also change the facial expression of at least one avatar (for example, the avatar 450 and the avatar 220, etc.) depicted in the isolation video according to a lip-sync method. To achieve this, the terminal device 20C can acquire voice data based on the speech of the user corresponding to the avatar. For example, the terminal device 20C can acquire voice data based on the speech of the user A in order to change the facial expression of the avatar 450, and can acquire voice data based on the speech of the second broadcaster in order to change the facial expression of the avatar 220. The method of acquiring such voice data is as described above in "5. Platform". The terminal device 20C can identify the joy, anger, sadness, and happiness of the speaker by analyzing the intonation and / or the speed of speech of the speaker based on the voice data acquired in this way. The terminal device 20C can reflect the joy, anger, sadness, and happiness identified in this way in the facial expression of the avatar displayed in the isolation video.

[0298] Also, in the various examples described above, the explanation has been given with a focus on a method (client rendering method) in which the terminal device 20C executes the operations in each step (ST1000 to ST1014). However, in at least one target step among ST1000 to ST1014, it is also possible to adopt a method (server rendering method) in which the operations related to generation of moving images that have been explained as being executed by the terminal device 20C are executed solely by one server device 30 or shared among multiple server devices 30. In this case, the one or more server devices 30 execute various operations related to the at least one target step instead of the terminal device 20C, thereby generating a first moving image and a quarantine moving image (including various user interfaces) and transmitting them to the terminal device 20C. As described above in "5. Platform", the server device 30 receives motion data and / or operation data from the terminal device 20 of each distributor, and can use the motion data and / or operation data instead of each terminal device 20 to generate various videos (including various processes including attaching an object to the avatar 450 or detaching an object from the avatar 450 in the first video and / or the isolation video, as described above in the modified example). Meanwhile, the server device 30 does not necessarily receive motion data from the terminal device 20 of each viewer. Therefore, when the server rendering method is adopted, the server device 30 may receive motion data from the terminal device of each viewer at unit time intervals or at any timing.

[0299] Also, in the various examples described above, the explanation has been given with a focus on a method (client rendering method) in which each terminal device 20C executes operations related to the generation of the first video, the isolation video, etc. However, the terminal device 20C can also adopt a method (browser rendering method) in which the terminal device 20C receives a web page from the server device 30 or the web server device 30 via a web browser and executes a computer program included in the web page to perform operations related to the generation of the first video, the isolation video, etc.

[0300] As described above, according to various embodiments, the terminal device 20C can display the isolated video by drawing the first virtual space including the user's avatar from a separate viewpoint disposed at a distance from the avatar, instead of a viewpoint disposed near the user's avatar (head) in the first video. This isolated video allows the user to objectively observe the appearance of his / her avatar (how it appears to other users, including each viewer and / or other distributors).

[0301] Furthermore, in a preferred embodiment, the terminal device 20C can prevent the avatar from rotating around a vertical axis in the isolated video, regardless of the contents of the motion data related to the user's motion. This isolated video allows the user to objectively observe the appearance of his / her avatar (how it appears to other users, including each viewer and / or other broadcasters) from a desired angle.

[0302] Furthermore, in a preferred embodiment, the terminal device 20C can display a user interface for attaching an object to an avatar or removing an object from an avatar in the isolated video. This allows the user to easily perform the task of attaching an object to an avatar or removing an object from an avatar while objectively observing the appearance of the avatar. Furthermore, the user can objectively observe the appearance of the avatar with or without an object attached.

[0303] The various technologies disclosed in the present application can be used not only in the aspect of providing videos between each distributor and each viewer, but also in the so-called "metaverse," a technology in which each user's avatar gathers and interacts in a virtual space. In other words, the various technologies disclosed in the present application can be used in the aspect in which each user's terminal device transmits data for operating the avatar to an information processing device (a server device or each user's terminal device) that generates various videos, and each user's terminal device generates (or receives) and displays various videos in which multiple users' avatars are arranged in a virtual space.

[0304] As would be easily understood by a person skilled in the art having the benefit of this disclosure, the various examples described above may be appropriately combined with each other in various patterns, unless a contradiction occurs.

[0305] In view of the many possible embodiments to which the principles of the invention disclosed herein may be applied, it is to be understood that the illustrated embodiments are merely preferred examples and should not be considered as limiting the scope of the claimed invention to these preferred examples. In fact, the scope of the claimed invention is defined by the appended claims. Accordingly, we claim the grant of a patent as our invention to all that falls within the scope of the claimed invention.

[0306] 8. Notes 8-1. Various aspects related to the platform

[0307] The computer program of the first aspect can be executed by at least one processor installed in a terminal device to cause the at least one processor to function as follows: when a first terminal device receives first movement data generated using a first tracking method based on the movement of a first broadcaster, generate a video including an animation of a first avatar of the first broadcaster based on the first movement data; when a second terminal device receives second movement data generated using a second tracking method different from the first tracking method based on the movement of a second broadcaster, generate a video including an animation of a second avatar of the second broadcaster based on the second movement data; when a second terminal device receives both the first movement data and the second movement data, generate a video including animation of the first avatar based on the first movement data and animation of the second avatar based on the second movement data, and display the video on a display unit.

[0308] In the computer program of the second aspect, the configuration of the first aspect described above can be adopted, in which "when both the first action data and the second action data are received, the first action data and the second action data received by the certain terminal device from the server device are data that the server device transmits commonly to the first terminal device, the second terminal device, and the certain terminal device."

[0309] In the computer program of the third aspect, the configuration of the first aspect described above can be adopted, in which "when both the first operation data and the second operation data are received, the first operation data is transmitted by the server device to the certain terminal device and the second terminal device, but not to the first terminal device, and the second operation data is transmitted by the server device to the certain terminal device and the first terminal device, but not to the second terminal device."

[0310] In the computer program of the fourth aspect, in any of the first to third aspects, the first tracking method can be configured as follows: "the first tracking method is a tracking method that generates motion data indicating at least the coordinates and angles of a device worn on the head and at least one controller worn or held on a part of the body other than the head, and the second tracking method is a face tracking method that generates motion data indicating at least the depth of each of a plurality of parts of the face and the coordinates and angle of the head."

[0311] In the computer program of the fifth aspect, the configuration of the fourth aspect described above can be adopted, in which "the certain terminal device is a terminal device that generates third action data using the first tracking method based on the action of a user who is a user of the certain terminal device."

[0312] In the computer program of the sixth aspect, the configuration of the fifth aspect can be adopted, in which "when both the first movement data and the second movement data are received, the at least one processor is caused to function so as to place the first avatar and the second avatar in a first virtual space as the moving image, and generate a first moving image represented from the viewpoint of the user himself / herself."

[0313] In the computer program of the seventh aspect, the configuration of the sixth aspect described above can be adopted, in which "the at least one processor is caused to function so as to change the viewpoint of the own user in the first video based on the third action data."

[0314] In the computer program of the eighth aspect, the configuration of the sixth or seventh aspect can be adopted, in which "the at least one processor is caused to function so as to generate, as the video, a second video by placing the first avatar and the second avatar in the first virtual space and representing the video from a certain viewpoint in the first virtual space."

[0315] In the computer program of the 9th aspect, the configuration of the 8th aspect described above can be adopted, in which "the at least one processor is caused to function so as to display the second moving image on a screen of a pseudo terminal device that is pseudo-represented as the first moving image."

[0316] In the computer program of the tenth aspect, the configuration of the eighth or ninth aspect can be adopted, in which "when the at least one processor determines that the user is a viewer watching the video and is executing a viewer mode, the at least one processor is caused to function so as not to draw the third avatar of the user in the first video and the second video."

[0317] In the computer program of the 11th aspect, the configuration of the 10th aspect can be adopted, in which "when it is determined that the at least one processor is executing the viewer mode, the third avatar of the user is not rendered in the video generated by a plurality of terminal devices including at least the first terminal device and the second terminal device."

[0318] In the computer program of the 12th aspect, the configuration of the 10th or 11th aspect can be adopted, in which "when the distance between the target avatar of either the first avatar or the second avatar in the first virtual space and the third avatar is equal to or less than a threshold, if the third action data indicates an action of moving toward the target avatar, the at least one processor is caused to function so as to disable the action."

[0319] In the computer program of the 13th aspect, in any of the 10th to 12th aspects, a configuration can be adopted in which "when the distance between a target avatar of either the first avatar or the second avatar in the first virtual space and the third avatar is equal to or less than a threshold, an object indicating the presence of the third avatar is displayed at the position of the third avatar that is not drawn in the video generated by a terminal device of the first terminal device or the second terminal device that corresponds to the target avatar and represented from the perspective of a target broadcaster of the first broadcaster or the second broadcaster that corresponds to the target avatar."

[0320] In the computer program of the 14th aspect, the configuration of the 8th or 9th aspect above can be adopted, where "when the at least one processor determines that the own user is a distributor that distributes the video in collaboration with the first distributor and the second distributor and is executing a collaborative distribution mode, the at least one processor is caused to function so as to transmit the third motion data to a server device and generate the first video including an animation of the first avatar based on the first motion data, an animation of the second avatar based on the second motion data, and an animation of the own user's third avatar based on the third motion data."

[0321] In the computer program of the 15th aspect, the configuration of the 14th aspect can be adopted, in which "when it is determined that the at least one processor is executing the collaborative distribution mode, the at least one processor is caused to function so as to generate the second video, the second video including an animation of the first avatar based on the first motion data, an animation of the second avatar based on the second motion data, and an animation of the third avatar based on the third motion data."

[0322] In the computer program of the 16th aspect, the configuration of the 15th aspect described above can be adopted, in which "when it is determined that the at least one processor is executing the collaborative distribution mode, the third avatar is rendered in the video generated by the first terminal device and the second terminal device."

[0323] In the computer program of the 17th aspect, in any of the 14th to 16th aspects, a configuration can be adopted in which "when the distance between a target avatar of either the first avatar or the second avatar in the first virtual space and the third avatar is equal to or less than a threshold, if the third action data indicates an action of moving toward the target avatar, the at least one processor is caused to function so as to disable the action."

[0324] In the computer program of the 18th aspect, in any of the 14th to 17th aspects, a configuration can be adopted in which "the at least one processor is caused to function so as to display on the display unit data indicating which of the first tracking method and the second tracking method is being used for each of the first distributor and the second distributor who distribute the video in collaboration with the user."

[0325] In the computer program of the 19th aspect, the configuration of the 15th aspect described above can be adopted, which "causes the at least one processor to function so that, in the second video, the second avatar, whose position in the first virtual space is fixed, is placed in a front row, and the first avatar and the third avatar, whose positions in the first virtual space change, are placed in a back row behind the front row."

[0326] In the computer program of the 20th aspect, the configuration of the 19th aspect above can be adopted, which "causes the at least one processor to function so as to position the first avatar and the third avatar in the back row at a higher position than the second avatar in the front row in the second video."

[0327] In the computer program of the 21st aspect, the configuration of the above-mentioned 4th aspect can be adopted, in which "the certain terminal device is a terminal device that generates fourth action data using the second tracking method based on the action of the user who is the user of the certain terminal device."

[0328] In the computer program of the 22nd aspect, the configuration of the 21st aspect can be adopted, in which "when the first movement data and the second movement data are received, the at least one processor is caused to function so as to place the first avatar and the second avatar in the first virtual space as the moving image, and generate a third moving image represented from a certain viewpoint in the first virtual space."

[0329] In the computer program of the 23rd aspect, the configuration of the 22nd aspect above can be adopted, in which "when the at least one processor determines that the own user is a distributor that distributes the video in collaboration with the first distributor and the second distributor and is executing a collaborative distribution mode, the at least one processor is caused to function so as to send the fourth motion data to the server device and generate the third video including an animation of the first avatar based on the first motion data, an animation of the second avatar based on the second motion data, and an animation of the own user's third avatar based on the fourth motion data."

[0330] In the computer program of the 24th aspect, in any of the 1st to 23rd aspects, the configuration may be such that "the at least one processor includes a central processing unit (CPU), a microprocessor, and / or a graphics processing unit (GPU)."

[0331] The method of the 25th aspect can be defined as "a method executed by at least one processor mounted on a terminal device that executes computer-readable instructions, wherein the at least one processor, by executing the instructions, includes: when a first terminal device receives first movement data generated using a first tracking method based on the movement of a first broadcaster, generating a video including an animation of a first avatar of the first broadcaster based on the first movement data; when a second terminal device receives second movement data generated using a second tracking method different from the first tracking method based on the movement of a second broadcaster, generating a video including an animation of a second avatar of the second broadcaster based on the second movement data; when a second terminal device receives both the first movement data and the second movement data, generating a video including an animation of the first avatar based on the first movement data and an animation of the second avatar based on the second movement data; and displaying the video on a display unit."

[0332] In the method of the 26th aspect, the configuration of the 25th aspect above can be adopted, in which "the at least one processor includes a central processing unit (CPU), a microprocessor, and / or a graphics processing unit (GPU)."

[0333] The computer program of the 27th aspect can cause the at least one processor to function as follows: "When executed by at least one processor installed in a terminal device, the at least one processor can cause the at least one processor to function as follows: first movement data generated by the terminal device using a first tracking method based on the movement of the user itself to be transmitted to a server device; second movement data generated by a second terminal device using a second tracking method different from the first tracking method based on the movement of a second broadcaster via the server device; generate a video including an animation of a first avatar of the user itself based on the first movement data and an animation of a second avatar of the second broadcaster based on the second movement data; and display the video on a display unit."

[0334] In the computer program of the 28th aspect, the configuration of the 27th aspect can be adopted, in which "the first tracking method is a tracking method that generates motion data indicating at least the coordinates and angles of a device worn on the head and at least one controller worn or held on a part of the body other than the head, and the second tracking method is a face tracking method that generates motion data indicating at least the depth of each of a plurality of parts of the face and the coordinates and angle of the head."

[0335] In the computer program of the 29th aspect, the configuration of the 27th aspect can be adopted, in which "the first tracking method is a face tracking method that generates motion data indicating at least the depth of each of a plurality of parts of the face and the coordinates and angle of the head, and the second tracking method is a tracking method that generates motion data indicating at least the coordinates and angle of a device worn on the head and at least one controller worn or held on a part of the body other than the head."

[0336] In the computer program of the 30th aspect, in any of the 27th to 29th aspects, the configuration may be adopted in which "the at least one processor includes a central processing unit (CPU), a microprocessor, and / or a graphics processing unit (GPU)."

[0337] The method of the 31st aspect can be "a method executed by at least one processor mounted on a certain terminal device that executes computer-readable instructions, the at least one processor executing the instructions to include transmitting first movement data generated by the certain terminal device using a first tracking method based on the movement of the user to a server device, receiving second movement data generated by a second terminal device using a second tracking method different from the first tracking method based on the movement of a second broadcaster via the server device, generating a video including an animation of a first avatar of the certain user based on the first movement data and an animation of a second avatar of the second broadcaster based on the second movement data, and displaying the video on a display unit."

[0338] In the method of the 32nd aspect, the configuration of the 31st aspect above can be adopted, in which "the at least one processor includes a central processing unit (CPU), a microprocessor, and / or a graphics processing unit (GPU)."

[0339] A server device according to a 33rd aspect of the present invention includes at least one processor, and when the at least one processor receives from a first terminal device first motion data generated by the first terminal device using a first tracking method based on a motion of a first distributor, the at least one processor transmits the first motion data to a certain terminal device to cause the certain terminal device to generate a video including an animation of a first avatar of the first distributor based on the first motion data, and transmits second motion data generated by a second terminal device using a second tracking method different from the first tracking method based on a motion of a second distributor to the certain terminal device to generate a video including an animation of a first avatar of the first distributor based on the first motion data. when the second movement data is received from a terminal device, the second movement data is transmitted to the certain terminal device, and a video including an animation of a second avatar of the second distributor based on the second movement data is generated by the certain terminal device; when both the first movement data and the second movement data are received, both the first movement data and the second movement data are transmitted to the certain terminal device, and a video including an animation of the first avatar based on the first movement data and an animation of the second avatar based on the second movement data is generated by the certain terminal device."

[0340] In the server device of the 34th aspect, the configuration in the 33rd aspect above can be adopted, in which "the at least one processor includes a central processing unit (CPU), a microprocessor, and / or a graphics processing unit (GPU)."

[0341] A method according to a 35th aspect is a method executed by at least one processor mounted on a server device that executes computer-readable instructions, the method including the steps of: when first motion data generated by a first terminal device using a first tracking method based on a motion of a first broadcaster is received from the first terminal device, transmitting the first motion data to a certain terminal device, and causing the certain terminal device to generate a video including an animation of a first avatar of the first broadcaster based on the first motion data; and causing a second terminal device to generate a video including an animation of a first avatar of the first broadcaster based on the first motion data, the video including an animation of a second avatar of the first broadcaster based on the first motion data. The method can include, when second motion data generated using a tracking method is received from the second terminal device, transmitting the second motion data to the certain terminal device and causing the certain terminal device to generate a video including an animation of a second avatar of the second distributor based on the second motion data, and, when both the first motion data and the second motion data are received, transmitting both the first motion data and the second motion data to the certain terminal device and causing the certain terminal device to generate a video including animation of the first avatar based on the first motion data and animation of the second avatar based on the second motion data.

[0342] In the method of the 36th aspect, the configuration of the 35th aspect above can be adopted, in which "the at least one processor includes a central processing unit (CPU), a microprocessor, and / or a graphics processing unit (GPU)."

[0343] The server device according to the 37th aspect can be configured as "a server device having at least one processor, the at least one processor being configured to transmit a web page including a computer program to a certain terminal device, the computer program being executed by the certain terminal device that has received the web page to cause the certain terminal device to function in the following manner: when a first terminal device receives first movement data generated using a first tracking method based on the movement of a first distributor, generate a video including an animation of a first avatar of the first distributor based on the first movement data; when a second terminal device receives second movement data generated using a second tracking method different from the first tracking method based on the movement of a second distributor, generate a video including an animation of a second avatar of the second distributor based on the second movement data; and when the certain terminal device receives both the first movement data and the second movement data, generate a video including animation of the first avatar based on the first movement data and animation of the second avatar based on the second movement data."

[0344] In the server device of the 38th aspect, the configuration in the 37th aspect above can be adopted, in which "the at least one processor includes a central processing unit (CPU), a microprocessor, and / or a graphics processing unit (GPU)."

[0345] 8-2. Various aspects related to input objects The computer program according to the first aspect "is executed by at least one processor mounted on a terminal device, and operates based on an action of a user who is a user of the terminal device, and displays a first moving image expressed from the viewpoint of the user, The at least one processor can be caused to function so as to "display an input object functioning as a user interface enabling input by the user in combination with the first moving image, and to vary the displayed input object depending on the type of the first moving image being displayed."

[0346] In the computer program of the second aspect, the configuration of the first aspect above can be adopted, in which "the at least one processor is caused to function so as to display the input object within the area in which the first video is displayed."

[0347] In the computer program of the third aspect, the configuration of the first aspect or the second aspect can be adopted, in which "the at least one processor is caused to function so as to display the input object movably in conjunction with a portion of the user's avatar included in the first video."

[0348] In the computer program of the fourth aspect, the configuration of the third aspect described above can be adopted, in which "the at least one processor is caused to function so as to display the input object as a pseudo-represented pseudo terminal device."

[0349] In a fifth aspect, in any one of the first to fourth aspects, the computer program further includes causing the at least one processor to function so as to display, as the first video, any one of four types of video, and further comprising: a first type of video to be displayed when the at least one processor determines that the user is in a preparation mode in which the user is neither a distributor nor a viewer, the first type of video being a first type of video in which an avatar of the user that moves based on the actions of the user in a virtual space is placed; and a second type of video to be displayed when the at least one processor determines that the user is in a solo distribution mode in which the user is a distributor, the first type of video being a first type of video in which an avatar of the user that moves based on the actions of the user in a virtual space is placed. a second type of video, which is displayed when the at least one processor determines that the user is in a co-distribution mode in which the user is a co-distributor, and in which an avatar of the user himself / herself that moves based on the actions of the user himself / herself and an avatar of at least one other distributor that moves based on the actions of the at least one other distributor in a virtual space; and a fourth type of video, which is displayed when the at least one processor determines that the user is in a viewing mode in which the user himself / herself is a viewer, and in which an avatar of the at least one distributor that moves based on the actions of the at least one distributor in a virtual space is displayed.

[0350] In the computer program of the sixth aspect, the configuration of the fifth aspect described above can be adopted, in which "the at least one processor is caused to function so as to display, as the first moving image, any one of the four types of moving images selected in accordance with an input by the user himself / herself to the input object."

[0351] In the computer program of the seventh aspect, the configuration of the sixth aspect above can be adopted, which is to "cause the at least one processor to function so that, when the fourth type of video is displayed as the first video, the input object is displayed in a first size, and, when the first type of video, the second type of video, or the third type of video is displayed as the first video, the input object is displayed in a second size larger than the first size."

[0352] In the computer program of the eighth aspect, the configuration of the seventh aspect described above can be adopted, in which "when the first type of video or the second type of video is displayed as the first video, the at least one processor is caused to function so as to include a second video in which the avatar of the user that moves based on the movement of the user in a virtual space is placed on the input object, and the avatar is displayed as viewed from the front."

[0353] In the computer program of the 9th aspect, the configuration of the 7th aspect above can be adopted, which "causes the at least one processor to function so that, when the third type of video is displayed as the first video, the input object includes a second video in which the avatar of the user that moves based on the actions of the user and the avatar of the at least one other broadcaster that moves based on the actions of the at least one other broadcaster are placed in a virtual space, and each avatar is displayed as seen from the front."

[0354] In the computer program of the tenth aspect, the configuration of the seventh aspect above can be adopted, which "causes the at least one processor to function so that, when the fourth type of video is displayed as the first video, an avatar of the at least one broadcaster that moves based on the movement of the at least one broadcaster in a virtual space is placed on the input object, and each avatar is displayed as viewed from the front, including a second video."

[0355] In the computer program of the 11th aspect, in any of the 8th to 10th aspects, the configuration can be adopted in which "the at least one processor is caused to function to generate the second video displaying each avatar as viewed from the position of a virtual camera placed in the virtual space used in the first video."

[0356] In the computer program of the 12th aspect, in any of the 8th to 11th aspects, a configuration can be adopted in which "when the first type of video, the second type of video or the third type of video is displayed as the first video, the at least one processor is caused to function so as to invert the avatar of the user himself and display it in the second video."

[0357] In the computer program of the 13th aspect, in any of the 8th to 12th aspects, a configuration can be adopted in which "when the second type of video or the third type of video is displayed as the first video, the second video included in the input object is identical to the video commonly displayed on a terminal device of at least one other viewer."

[0358] In the computer program of the 14th aspect, in any of the 8th to 13th aspects, a configuration can be adopted in which "the at least one processor is caused to function so as to display an icon selected by the user in the input object in order to determine which of the first type of video, the second type of video, the third type of video, and the fourth type of video should be displayed as the first video."

[0359] In the computer program of the 15th aspect, in any of the 8th to 14th aspects, a configuration can be adopted in which "the at least one processor is caused to function so that, when the second type of video, the third type of video, or the fourth type of video is displayed as the first video, a comment sent by a viewer to any of the distributors is displayed in the first video in a background located behind the input object."

[0360] In the computer program of the 16th aspect, in any of the 8th to 15th aspects, the configuration can be adopted, which "causes the at least one processor to function as follows: when displaying the fourth type of video as the first video, display a plurality of gift objects to the side, above or below an avatar of the at least one broadcaster in the fourth type of video, determine one of the plurality of gift objects selected by the user himself as a gift object to be sent to the at least one broadcaster, display the determined gift object in the fourth type of video, and display the determined gift object in the second video."

[0361] In the computer program of the 17th aspect, in any of the first to 16th aspects, the configuration may be such that "the at least one processor includes a central processing unit (CPU), a microprocessor, and / or a graphics processing unit (GPU)."

[0362] The method of the 18th aspect can be "a method executed by at least one processor mounted on a terminal device that executes computer-readable instructions, the at least one processor executing the instructions to operate based on the actions of a user who is a user of the terminal device and display a first moving image represented from the perspective of the user, displaying an input object that functions as a user interface allowing input by the user in combination with the first moving image, and changing the displayed input object depending on the type of the first moving image displayed."

[0363] In the method of the 19th aspect, the configuration of the 20th aspect above can be adopted, in which "the at least one processor includes a central processing unit (CPU), a microprocessor, and / or a graphics processing unit (GPU)."

[0364] A system according to the 20th aspect can be "a system including a terminal device and a server device connected to the terminal device via a communication line, wherein the terminal device is configured to transmit to the server device motion data generated based on motion of a user of the terminal device, operate based on the motion data, display a first moving image represented from the user's perspective, display input objects functioning as a user interface enabling input by the user in combination with the first moving image, and change the displayed input objects depending on the type of the first moving image displayed, and the server device is configured to transmit the motion data received from the terminal device to at least one other terminal device."

[0365] The system according to the twenty-first aspect can employ the configuration of the twenty-sixth aspect above, in which "the communication line includes the Internet."

[0366] The method according to the 22nd aspect can be "a method executed by a system including a terminal device and a server device connected to the terminal device via a communication line, comprising: the terminal device transmitting to the server device motion data generated based on motion of a user of the terminal device; the terminal device operating based on the motion data and displaying a first moving image represented from the user's perspective; the terminal device displaying, in combination with the first moving image, an input object functioning as a user interface enabling input by the user; the terminal device changing the displayed input object depending on the type of the first moving image displayed; and the server device transmitting the motion data received from the terminal device to at least one other terminal device."

[0367] In the method according to the twenty-third aspect, the configuration of the twenty-second aspect can be adopted in which "the communication line includes the Internet."

[0368] The server device according to the 24th aspect can be described as "a server device having at least one processor, the at least one processor being configured to transmit a web page including a computer program to a terminal device, the computer program being executed by the terminal device receiving the web page, causing the terminal device to function in the following manner: operate based on the actions of a user of the terminal device, display a first moving image represented from the user's viewpoint, display input objects functioning as a user interface enabling input by the user in combination with the first moving image, and change the displayed input objects depending on the type of the first moving image displayed."

[0369] In the server device of the 25th aspect, the configuration in the 24th aspect above can be adopted, in which "the at least one processor includes a central processing unit (CPU), a microprocessor, and / or a graphics processing unit (GPU)."

[0370] 8-3. Various aspects related to closet mode The computer program of the first aspect can cause the at least one processor to function as follows: "When executed by at least one processor mounted on a terminal device, the at least one processor can acquire first movement data generated based on the movement of a user of the terminal device using multiple tracking devices worn by the user, who is a user of the terminal device; operate based on the first movement data and display a first video that represents a virtual space as viewed from the user's viewpoint; operate based on the first movement data and display a second video that represents the virtual space including the user's avatar as viewed from an isolated viewpoint at a distance from the user's avatar; and attach an object to the user's avatar or detach an object from the user's avatar in accordance with operation data generated based on the operation of the user."

[0371] In the computer program of the second aspect, the configuration of the first aspect described above can be adopted, which "causes the at least one processor to function to switch between a first mode in which the first video is displayed in a certain area and a second mode in which the second video is displayed in the certain area, or to display the first video and the second video in parallel in separate areas."

[0372] In the computer program of the third aspect, in the first or second aspect described above, a configuration can be adopted in which "the at least one processor is caused to function so as to use the viewpoint of the user's avatar placed in the virtual space as the viewpoint of the user's own user."

[0373] In the computer program of the fourth aspect, in any of the first to third aspects, the at least one processor may be configured to "function so as to change the viewpoint of the own user and to move an avatar of the own user in the first video based on the first action data, and to move the avatar of the own user in the second video based on the first action data, and not to change the isolated viewpoint regardless of the first action data."

[0374] In the computer program according to the fifth aspect, in any one of the first to fourth aspects, "in the second video, regardless of the first action data, the avatar of the user is not rotated around a rotation axis extending vertically in the virtual space; The at least one processor may be configured to function in such a manner as to

[0375] In the computer program of the sixth aspect, in any of the first to fifth aspects, a configuration can be adopted in which "the at least one processor is caused to function so as not to change the direction in which the head of the user's avatar faces in the virtual space in the second video, regardless of the first motion data."

[0376] In the computer program of the seventh aspect, in any of the first to sixth aspects, a configuration can be adopted in which "the at least one processor is caused to function so as not to change the position of the avatar of the user himself in the virtual space in the second video, regardless of the first motion data."

[0377] In the computer program of the eighth aspect, the configuration of the fifth aspect above can be adopted, in which "the plurality of tracking devices include a right hand controller operated by the right hand of the user and a left hand controller operated by the left hand of the user, and the at least one processor is caused to function so as to display a specific user interface in the second video, and to rotate the avatar of the user around the rotation axis in the second video in accordance with the operation by the user of the user of the user interface, the right hand controller, and / or the left hand controller."

[0378] In a computer program according to a ninth aspect, in any one of the first to seventh aspects, "the plurality of tracking devices include a right hand controller operated by the right hand of the user and a left hand controller operated by the left hand of the user, and a specific user interface is displayed in the second video; The at least one processor may be configured to "change the isolated viewpoint in the second video in accordance with operations by the user on the user interface, the right hand controller, and / or the left hand controller."

[0379] In the computer program of the tenth aspect, in any of the first to ninth aspects, a configuration can be adopted in which "first voice data based on the speech of the user himself / herself, and in the second video, the at least one processor is caused to function so as to change the facial expression of the avatar of the user himself / herself based on the first voice data."

[0380] In the computer program of the 11th aspect, in any of the first to tenth aspects, a configuration can be adopted in which "the first video includes an avatar of the other user moving in accordance with second movement data generated based on the movement of the other user using a plurality of tracking devices attached to the other user."

[0381] In the computer program of the 12th aspect, the configuration of the 11th aspect described above can be adopted, in which "the at least one processor is caused to function so as to use the viewpoint of the avatar of the other user placed in the virtual space as the isolated viewpoint."

[0382] In the computer program of the thirteenth aspect, the configuration of the eleventh or twelfth aspect above can be adopted in which "the second video is not displayed on the terminal device of the other user."

[0383] In the computer program of the 14th aspect, in any of the first to 13th aspects, the configuration can be adopted in which "the at least one processor is caused to function so as to cause another terminal device receiving the first movement data from the server device to display a third video representing the virtual space including an avatar of the user of the other terminal device moving in accordance with the first movement data as viewed from the perspective of the user of the other terminal device by transmitting the first movement data to a server device."

[0384] In the computer program of the 15th aspect, the configuration of the 14th aspect described above can be adopted, in which "the at least one processor is caused to function so as to use the viewpoint of the user of the other terminal device as the isolated viewpoint."

[0385] In the computer program according to the sixteenth aspect, the configuration in the fourteenth or fifteenth aspect can be adopted in which "the second moving image is not displayed on the other terminal device."

[0386] In the computer program of the 17th aspect, in any of the first to 16th aspects, the configuration may be such that "in the second video, a plurality of objects are displayed to the side, above or below the user's avatar, and any one of the plurality of objects selected by the user is determined as an additional object to be attached to the user's avatar, and in the second video, the user's avatar with the determined additional object attached is displayed."

[0387] In the computer program of the 18th aspect, the configuration of the 17th aspect can be adopted, in which "the at least one processor is caused to function so as to determine, as the additional object, an object among the plurality of objects that is moved in the second video by an instruction object operating in accordance with the first action data until it comes into contact with the user's avatar."

[0388] In the computer program of the 19th aspect, in the first to third aspects, the configuration can be adopted in which "the at least one processor is caused to function so as to change the viewpoint of the own user and to move an avatar of the own user in the first video based on the first movement data, and to move an avatar of the own user in the second video regardless of the first movement data."

[0389] In the computer program of the twentieth aspect, in any of the first to nineteenth aspects, a configuration can be adopted in which "the multiple tracking devices worn by the user are attached to the right hand, left hand, and head of the user, respectively."

[0390] In the computer program of the 21st aspect, in any of the first to 18th aspects, the configuration can be adopted which "causes the at least one processor to function so as to generate and display the first moving image based on the first motion data, and to generate and display the second moving image based on the first motion data."

[0391] In the computer program of the 22nd aspect, in any of the 1st to 21st aspects, a configuration can be adopted in which "the at least one processor is caused to function so as to receive from the server device and display the first video generated by the server device, and to receive from the server device and display the second video generated by the server device."

[0392] In the computer program of the 23rd aspect, in any of the 1st to 22nd aspects, the configuration may be adopted in which "the at least one processor includes a central processing unit (CPU), a microprocessor, and / or a graphics processing unit (GPU)."

[0393] The method of the 24th aspect can be "a method executed by at least one processor mounted on a terminal device that executes computer-readable instructions, the at least one processor executing the instructions to obtain first movement data generated based on a movement of a user of the terminal device using a plurality of tracking devices attached to the user, the user being a user of the terminal device; displaying a first video that operates based on the first movement data and represents a virtual space as viewed from the viewpoint of the user; displaying a second video that operates based on the first movement data and represents the virtual space including the user's avatar as viewed from a remote viewpoint at a distance from the user's avatar; and attaching an object to the user's avatar or detaching an object from the user's avatar in accordance with operation data generated based on the operation of the user."

[0394] In the method of the 25th aspect, the configuration of the 24th aspect above can be adopted, in which "the at least one processor includes a central processing unit (CPU), a microprocessor, and / or a graphics processing unit (GPU)."

[0395] A server device according to the 26th aspect can be "a server device having at least one processor, the at least one processor being configured to: acquire motion data generated based on the motion of a user of a terminal device using a plurality of tracking devices worn by the user of the terminal device, operate based on the motion data, generate a first video that represents a virtual space as seen from the user's viewpoint, and transmit the first video to the terminal device; operate based on the motion data, generate a second video that represents the virtual space including the user's avatar as seen from an isolated viewpoint at a distance from the user's avatar, and transmit the second video to the terminal device; receive operation data generated based on the user's operation from the terminal device; and attach an object to the user's avatar or detach an object from the user's avatar based on the operation data."

[0396] In the server device of the 27th aspect, the configuration in the 26th aspect above can be adopted, in which "the at least one processor includes a central processing unit (CPU), a microprocessor, and / or a graphics processing unit (GPU)."

[0397] The method of the 28th aspect can be "a method executed by at least one processor mounted on a server device that executes computer-readable instructions, the at least one processor executing the instructions to obtain motion data generated based on motion of a user of a terminal device using a plurality of tracking devices worn by the user of the terminal device, generating and transmitting to the terminal device a first video that operates based on the motion data and depicts a virtual space as seen from the user's viewpoint, generating and transmitting to the terminal device a second video that operates based on the motion data and depicts the virtual space including the user's avatar as seen from a remote viewpoint at a distance from the user's avatar, receiving from the terminal device operation data generated based on the user's operation, and attaching an object to the user's avatar or detaching an object from the user's avatar based on the operation data."

[0398] In the method of the 29th aspect, the configuration of the 28th aspect above can be adopted, in which "the at least one processor includes a central processing unit (CPU), a microprocessor, and / or a graphics processing unit (GPU)."

[0399] A system according to a 30th aspect can be a system including "a terminal device and a server device connected to the terminal device via a communication line, wherein the terminal device is configured to: acquire motion data generated based on the motion of a user of the terminal device using a plurality of tracking devices worn by the user of the terminal device, operate based on the motion data, display a first video representing a virtual space as viewed from the user's viewpoint, operate based on the motion data, display a second video representing the virtual space including the user's avatar as viewed from a remote viewpoint at a distance from the user's avatar, and attach an object to the user's avatar or detach an object from the user's avatar in accordance with operation data generated based on the user's operation; and the server device is configured to transmit the motion data received from the terminal device to at least one other terminal device."

[0400] In the system according to the thirty-first aspect, the configuration of the thirtieth aspect described above can be adopted in which "the communication line includes the Internet."

[0401] The method according to the 32nd aspect can be "a method executed by a system including a terminal device and a server device connected to the terminal device via a communication line, the method including: the terminal device acquiring motion data generated based on motion of the user of the terminal device using a plurality of tracking devices worn by the user of the terminal device; the terminal device operating based on the motion data and displaying a first video depicting a virtual space as viewed from the user's viewpoint; the terminal device operating based on the motion data and displaying a second video depicting the virtual space including the user's avatar as viewed from a remote viewpoint at a distance from the user's avatar; attaching an object to the user's avatar or detaching an object from the user's avatar in accordance with operation data generated based on the user's operation; and the server device transmitting the motion data received from the terminal device to at least one other terminal device."

[0402] In the method according to the 33rd aspect, the configuration of the 32nd aspect can be adopted in which "the communication line includes the Internet."

[0403] The server device according to the 34th aspect is "a server device having at least one processor, the at least one processor being configured to transmit a web page including a computer program to a terminal device, the computer program being executed by the terminal device receiving the web page, to cause the terminal device to function as follows: acquire motion data generated based on motion of the user using a plurality of tracking devices worn by a user of the terminal device, operate based on the motion data, display a first video representing a virtual space as viewed from the user's viewpoint, operate based on the motion data, display a second video representing the virtual space including the user's avatar as viewed from a remote viewpoint at a distance from the user's avatar, and attach an object to the user's avatar or detach an object from the user's avatar in accordance with operation data generated based on the user's operation."

[0404] In the server device of the 35th aspect, the configuration in the 34th aspect above can be adopted, in which "the at least one processor includes a central processing unit (CPU), a microprocessor, and / or a graphics processing unit (GPU)." [Explanation of symbols]

[0405] 1. Communication Systems 20, 20A, 20B, 20C Terminal equipment 30, 30A, 30B, 30C Server device

Claims

1. When executed by at least one processor mounted on a terminal device, A first video that represents a predetermined virtual space from the viewpoint of an avatar of a user of the terminal device, and / or a second video viewed from a position of a virtual camera arranged in the virtual space, the second video being different from the first video; activating the at least one processor to display a screen including The second video is an avatar of the user that is placed in the virtual space, and displays the avatar at least viewed from the front. A computer program comprising:

2. an input object that functions as a user interface that enables input by the user is displayed on the screen; The computer program product of claim 1 , wherein the second animation is displayed within the input object.

3. 3. The computer program product according to claim 2, wherein the input by the user includes an input for selecting a video to be displayed on the screen.

4. The computer program product according to claim 1 , wherein the video includes one of the first video and the second video, in response to an input by the user.

5. When the second moving image is displayed on the moving image, The computer program according to claim 1 , further comprising: rotating the avatar of the user himself / herself around a predetermined rotation axis in accordance with an operation by the user himself / herself.

6. When the second moving image is displayed on the moving image, 2. The computer program according to claim 1, wherein the position of the avatar of the user is not changed in accordance with the operation by the user.

7. 4. The computer program product according to claim 3, wherein the input for selecting the moving image to be displayed on the screen is a selection of one of a plurality of display modes.

8. According to the input by the user, displaying, as the first video, any one of four types of videos represented from the viewpoint of the user's avatar; displaying, as the second moving image, a moving image expressed as viewed from the position of a virtual camera disposed in the virtual space used in the first moving image; 4. The computer program product of claim 3, further comprising:

9. The computer program product of claim 2, characterized in that the input by the user includes an input for selecting a set of videos to be displayed on the screen, a video to be displayed as the first video, and a video to be displayed as the second video.

10. 2. The computer program product of claim 1, further comprising: causing the at least one processor to: display the input object as a simulated pseudo-terminal device.

11. The computer program product of claim 1 , wherein the at least one processor comprises a central processing unit (CPU), a microprocessor, and / or a graphics processing unit (GPU).

12. 1. A method performed by at least one processor in a terminal device executing computer readable instructions, comprising: The at least one processor executes the instructions to: A first video that represents a predetermined virtual space from the viewpoint of an avatar of a user of the terminal device, and / or a second video viewed from a position of a virtual camera arranged in the virtual space, the second video being different from the first video; displaying a screen including The second video is an avatar of the user that is placed in the virtual space, and displays the avatar at least viewed from the front. A method comprising:

13. The method of claim 12 , wherein the at least one processor comprises a central processing unit (CPU), a microprocessor, and / or a graphics processing unit (GPU).

14. A system including a terminal device and a server device connected to the terminal device via a communication line, The terminal device A first video representing a predetermined virtual space from the viewpoint of an avatar of a user of the terminal device, and / or a second video viewed from a position of a virtual camera arranged in the virtual space, the second video being different from the first video; configured to display a screen including The second video is an avatar of the user that is placed in the virtual space, and displays the avatar at least viewed from the front. A system characterized by:

15. The system of claim 14 , wherein the communication line includes the Internet.

16. A method executed by a system including a terminal device and a server device connected to the terminal device via a communication line, comprising: A first video representing a predetermined virtual space from the viewpoint of an avatar of a user of the terminal device, and / or a second video viewed from a position of a virtual camera arranged in the virtual space, the second video being different from the first video; displaying a screen including The second video is an avatar of the user that is placed in the virtual space, and displays the avatar at least viewed from the front. A method comprising:

17. The method of claim 16 , wherein the communication line comprises the Internet.

18. A server device including at least one processor, The at least one processor is configured to transmit a web page including a computer program to a terminal device; The computer program comprising: The web page is executed by the terminal device that has received the web page, A first video representing a predetermined virtual space from the viewpoint of an avatar of a user of the terminal device, and / or a second video viewed from a position of a virtual camera arranged in the virtual space, the second video being different from the first video; and causing the terminal device to display a screen including The second video is an avatar of the user that is placed in the virtual space, and displays the avatar at least viewed from the front. A server device comprising:

19. 20. The server apparatus of claim 18, wherein the at least one processor comprises a central processing unit (CPU), a microprocessor, and / or a graphics processing unit (GPU).

Images