program

Abstract

To provide a program, a method, and an information processing device capable of urging a user to be continuously involved in content.SOLUTION: A video in a virtual space in which a plurality of kinds of objects including a character object that moves on the basis of first data common between a plurality of computers are arranged is displayed in a first part, a video including the character object is displayed on the basis of second data in a second part, the second data is data including information for making the character object issue a voice, and is externally transmitted to one or more arbitrary computers including a first computer used by a first user, and has utterance content different from data to be transmitted to a second user used by a second user, a specific object to be displayed in the first part is changeable in a display mode, and a display mode of the specific object to be displayed in the second part becomes a display mode that changes in the first part.SELECTED DRAWING: Figure 30

Description

[Technical Field]

[0001] This disclosure is directed to Mu Regarding. [Background technology]

[0002] Non-Patent Document 1 discloses a technique for allowing viewers to participate in live streaming by characters. [Prior art documents] [Non-patent literature]

[0003] [Non-Patent Document 1] GF, "What is the secret behind the cuteness of the popular virtual beauty "Shinonome Megu"?", [online], March 6, 2018, MoguLive, [Retrieved February 19, 2019], Internet<https: / / www.moguravr.com / shinonome-megu / > Summary of the Invention [Problem to be solved by the invention]

[0004] Conventional technology has room to encourage viewers to continue participating in content.

[0005] The present invention has been conceived in view of the above circumstances, and its object is to provide a program that can encourage users to continuously participate in content. M The purpose is to provide. [Means for solving the problem]

[0006] According to one aspect of an embodiment presented herein, On the computer, In the first part, the data received from the outside Complex The system operates based on common first data among a plurality of computers used by a plurality of users. No. 1a step of displaying an image of a virtual space in which a plurality of types of objects including the object are arranged; and in a second part, , 1st displaying a video including the object; Realize The second data is Data transmitted from outside to a computer, Object Message by The data includes the law of nature, A specific object among the multiple types of objects that can be displayed in the first part is an object whose display manner can be changed, and the display manner of the specific object that can be displayed in the second part is the display manner of the specific object that changes in the first part. ,program . [Effects of the Invention]

[0007] According to the present invention, it is possible to encourage users to continuously participate in content. [Brief explanation of the drawings]

[0008] [Figure 1] FIG. 1 is a diagram illustrating an example of an overview of a system according to an embodiment. [Figure 2] FIG. 2 is a block diagram showing an example of a hardware configuration of a user terminal according to an embodiment. [Figure 3] FIG. 2 is a block diagram illustrating an example of a hardware configuration of a server according to an embodiment. [Figure 4] FIG. 1 is a block diagram showing an example of a hardware configuration of a game playing terminal according to an embodiment. [Figure 5] FIG. 2 is a block diagram showing an example of a hardware configuration of a distribution terminal according to an embodiment. [Figure 6] FIG. 2 is a block diagram showing an example of a functional configuration of a user terminal, a server, and an HMD set according to an embodiment. [Figure 7] FIG. 2 is a block diagram illustrating an example of a functional configuration of a distribution terminal according to an embodiment. [Figure 8]10 is a flowchart illustrating a portion of the processing executed in a user terminal and a game playing terminal according to an embodiment. [Figure 9] 1A and 1B are diagrams illustrating an example of a virtual space provided to a player and a field of view image viewed by the player according to an embodiment. [Figure 10] A figure showing an example of a virtual space provided to a user of a user terminal and a field of view image viewed by the user, according to one embodiment. [Figure 11] FIG. 10 is a diagram showing another example of a field of view image visually recognized by a user of a user terminal. [Figure 12] FIG. 10 is a diagram showing yet another example of a field of view image visually recognized by a user of a user terminal. [Figure 13] 10 is a flowchart illustrating a part of a process executed in a game playing terminal according to an embodiment. [Figure 14] 10 is a flowchart illustrating a part of a process executed in a user terminal according to an embodiment. [Figure 15] 10 is a flowchart illustrating a part of a process executed in a server according to an embodiment. [Figure 16] FIG. 1 illustrates an example of a list of users who have participated in a game, according to one embodiment. [Figure 17] 10 is a flowchart illustrating a part of a process executed in a distribution terminal according to an embodiment. [Figure 18] FIG. 10 shows a specific example of a screen displayed on a distribution terminal according to an embodiment. [Figure 19] FIG. 10 is a diagram showing another specific example of a screen displayed on a distribution terminal according to an embodiment. [Figure 20] FIG. 10 illustrates one example of voice input by a player, according to one embodiment. [Figure 21] FIG. 10 shows yet another specific example of a screen displayed on a distribution terminal according to an embodiment and an outline of distribution of operation instruction data. [Figure 22]FIG. 10 illustrates another example of voice input by a player, according to one embodiment. [Figure 23] FIG. 10 shows yet another specific example of a screen displayed on a distribution terminal according to an embodiment and an outline of distribution of operation instruction data. [Figure 24] FIG. 10 is a diagram illustrating an overview of transmission of game progress information from a game playing terminal to a user terminal according to an embodiment. [Figure 25] 10 is a flowchart illustrating a part of a process executed in a user terminal according to an embodiment. [Figure 26] FIG. 10 is a diagram showing a specific example of video playback. [Figure 27] FIG. 10 is a diagram showing another specific example of video playback. [Figure 28] FIG. 10 is a diagram showing an example of a screen displayed on a user terminal during a live distribution part. [Figure 29] FIG. 10 is a diagram showing an example of a display on a recording screen during a recording part. [Figure 30] FIG. 10 is a diagram showing an example of a display screen during a playback part. [Figure 31] FIG. 10 is a diagram showing an example of a display screen during a playback part for playing back an individual message with a special benefit. [Figure 32] 10 is a flowchart showing an example of a processing flow from the end of a live distribution part through a recording part to a playback part. DETAILED DESCRIPTION OF THE INVENTION

[0009] The system according to the present disclosure is a system for providing a game to multiple users. The system will be described below with reference to the drawings. Note that the present invention is not limited to these examples, but is defined by the claims, and all modifications within the meaning and scope equivalent to the claims are intended to be included in the present invention. In the following description, the same elements in the drawings will be given the same reference numerals, and redundant explanations will not be repeated.

[0010] <System 1 Operation Overview> FIG. 1 is a diagram illustrating an overview of a system 1 according to this embodiment. The system 1 includes multiple user terminals 100 (computers), a server 200, a game play terminal 300 (external device, second external device), and a distribution terminal 400 (external, first external device). Note that FIG. 1 illustrates user terminals 100A-100C, or in other words, three user terminals 100, as an example of the multiple user terminals 100; however, the number of user terminals 100 is not limited to the illustrated example. In this embodiment, when there is no need to distinguish between the user terminals 100A-C, they will be referred to as "user terminals 100." The user terminals 100, game play terminals 300, and distribution terminal 400 are connected to the server 200 via a network 2. The network 2 is comprised of the Internet and various mobile communication systems established by wireless base stations (not shown). Examples of such mobile communication systems include so-called 3G and 4G mobile communication systems, LTE (Long Term Evolution), and wireless networks (such as Wi-Fi (registered trademark)) that can be connected to the Internet via a predetermined access point.

[0011] (Game Overview) In this embodiment, a game that is primarily played by a user of a game play terminal 300 will be described as an example of a game provided by the system 1 (hereinafter referred to as the game). Hereinafter, a user of a game play terminal 300 will be referred to as a "player." As an example, a player (performer) progresses through the game by operating a character that appears in the game. Furthermore, in the game, a user of a user terminal 100 plays a role in supporting the player's progress through the game. Details of the game will be described later. Note that the game provided by the system 1 may be any game in which multiple users participate, and is not limited to this example.

[0012] (Game play terminal 300) The game play terminal 300 progresses the game in response to input operations by the player. The game play terminal 300 also sequentially distributes information generated by the player's game play (hereinafter referred to as game progress information) to the server 200 in real time.

[0013] (Server 200) The server 200 transmits game progress information received in real time from the game play terminal 300 to the user terminal 100. The server 200 also mediates the transmission and reception of various types of information between the user terminal 100, the game play terminal 300, and the distribution terminal 400.

[0014] (Distribution terminal 400) The distribution terminal 400 generates action instruction data in response to an input operation by the user of the distribution terminal 400, and distributes the action instruction data to the user terminal 100 via the server 200. The action instruction data is data for playing a video on the user terminal 100, and more specifically, data for causing a character appearing in the video to perform a certain action.

[0015] In this embodiment, as an example, the user of the distribution terminal 400 is a player of the game. Also, as an example, the video played on the user terminal 100 based on the action instruction data is a video of a character operated by the player in the game moving. "Movement" refers to moving at least a part of the character's body, and includes speaking. Therefore, the action instruction data according to this embodiment includes, for example, voice data for making the character speak and motion data for moving the character's body.

[0016] As an example, the action instruction data is transmitted to the user terminal 100 after the game has ended. Details of the action instruction data and the video played based on the action instruction data will be described later.

[0017] (User terminal 100) The user terminal 100 receives game progress information in real time and generates and displays a game screen using that information. In other words, the user terminal 100 reproduces the game screen of the game being played by the player through real-time rendering. This allows the user of the user terminal 100 to view the same game screen as the player views while playing the game, at approximately the same time as the player.

[0018] Furthermore, the user terminal 100 generates information to assist the player in progressing through the game in response to input operations by the user, and transmits this information to the game play terminal 300 via the server 200. Details of this information will be described later.

[0019] Furthermore, the user terminal 100 receives the action instruction data from the distribution terminal 400, and generates and plays back a moving image (video) using the action instruction data. In other words, the user terminal 100 renders and plays back the action instruction data.

[0020] <System 1 hardware configuration> Fig. 2 is a diagram showing the hardware configuration of a user terminal 100. Fig. 3 is a diagram showing the hardware configuration of a server 200. Fig. 4 is a diagram showing the hardware configuration of a game playing terminal 300. Fig. 5 is a diagram showing the hardware configuration of a distribution terminal 400.

[0021] (User terminal 100) In this embodiment, an example will be described in which the user terminal 100 is realized as a smartphone, but the user terminal 100 is not limited to a smartphone. For example, the user terminal 100 may be realized as a feature phone, a tablet computer, a laptop computer (a so-called notebook computer), or a desktop computer. The user terminal 100 may also be a game device suitable for playing games.

[0022] 2, the user terminal 100 includes a processor 10, a memory 11, a storage 12, a communication interface (IF) 13, an input / output IF 14, a touch screen 15 (display unit), a camera 17, and a distance measurement sensor 18. These components of the user terminal 100 are electrically connected to each other via a communication bus. Note that instead of or in addition to the touch screen 15, the user terminal 100 may include an input / output IF 14 to which a display (display unit) configured separately from the user terminal 100 main body can be connected.

[0023] 2, the user terminal 100 may be configured to be able to communicate with one or more controllers 1020. The controller 1020 establishes communication with the user terminal 100 in accordance with a communication standard such as Bluetooth (registered trademark). The controller 1020 may have one or more buttons, etc., and transmits output values ​​based on user input operations on the buttons, etc., to the user terminal 100. The controller 1020 may also have various sensors, such as an acceleration sensor and an angular velocity sensor, and transmits output values ​​of the various sensors to the user terminal 100.

[0024] Instead of or in addition to the user terminal 100 being provided with the camera 17 and the distance measurement sensor 18, the controller 1020 may have the camera 17 and the distance measurement sensor 18.

[0025] For example, at the start of a game, the user terminal 100 preferably prompts the user using the controller 1020 to input user identification information such as the user's name or login ID via the controller 1020. This enables the user terminal 100 to associate the controller 1020 with the user, and to identify which user the received output value belongs to based on the sender (controller 1020) of the output value.

[0026] When a user terminal 100 communicates with multiple controllers 1020, each user holds a controller 1020, allowing a single user terminal 100 to realize a multiplay mode without communicating with other devices such as the server 200 via the network 2. Alternatively, the user terminals 100 may connect to each other via a wireless standard such as a wireless local area network (LAN) standard (connecting to each other via a wireless standard without going through the server 200) to realize a multiplay mode locally using multiple user terminals 100. When a single user terminal 100 realizes the above-described multiplay mode locally, the user terminal 100 may further include at least some of the various functions of the server 200, which will be described later. When a plurality of user terminals 100 realize the above-described multiplay mode locally, the plurality of user terminals 100 may include the various functions of the server 200, which will be described later, in a distributed manner.

[0027] Even when the above-described multiplay is realized locally, the user terminal 100 may communicate with the server 200. For example, information indicating the results of a game, such as the score or win / loss, may be associated with user identification information and transmitted to the server 200.

[0028] The controller 1020 may also be configured to be detachable from the user terminal 100. In this case, a coupling unit for the controller 1020 may be provided on at least one surface of the housing of the user terminal 100. When the user terminal 100 and the controller 1020 are coupled to each other via the coupling unit by a wire, the user terminal 100 and the controller 1020 transmit and receive signals via the wire.

[0029] 2, the user terminal 100 may accept the attachment of a storage medium 1030 such as an external memory card via the input / output IF 14. This allows the user terminal 100 to read programs and data recorded on the storage medium 1030. The program recorded on the storage medium 1030 is, for example, a game program.

[0030] The user terminal 100 may store in the memory 11 of the user terminal 100 a game program acquired by communicating with an external device such as the server 200, or may store in the memory 11 a game program acquired by reading it from the storage medium 1030.

[0031] As described above, the user terminal 100 includes the communication IF 13, the input / output IF 14, the touch screen 15, the camera 17, and the distance measurement sensor 18 as examples of mechanisms for inputting information to the user terminal 100. Each of the above-described units as input mechanisms can be considered as an operation unit configured to accept input operations by the user.

[0032] For example, when the operation unit is configured with at least one of the camera 17 and the distance measurement sensor 18, the operation unit detects an object 1010 near the user terminal 100 and identifies an input operation from the detection result of the object. As an example, a user's hand, a marker of a predetermined shape, or the like is detected as the object 1010, and the input operation is identified based on the color, shape, movement, or type of the object 1010 obtained as the detection result. More specifically, when the user's hand is detected from an image captured by the camera 17, the user terminal 100 identifies and accepts a gesture (a series of movements of the user's hand) detected based on the captured image as the user's input operation. Note that the captured image may be a still image or a video.

[0033] Alternatively, if the operation unit is configured with the touch screen 15, the user terminal 100 identifies and accepts a user operation performed on the input unit 151 of the touch screen 15 as the user's input operation. Alternatively, if the operation unit is configured with the communication IF 13, the user terminal 100 identifies and accepts a signal (e.g., an output value) transmitted from the controller 1020 as the user's input operation. Alternatively, if the operation unit is configured with the input / output IF 14, the user terminal 100 identifies and accepts a signal output from an input device (not shown) connected to the input / output IF 14 that is different from the controller 1020 as the user's input operation.

[0034] (Server 200) The server 200 may be, for example, a general-purpose computer such as a workstation or a personal computer. The server 200 includes a processor 20, a memory 21, a storage 22, a communication IF 23, and an input / output IF 24. These components of the server 200 are electrically connected to each other by a communication bus.

[0035] (Game play terminal 300) The game play terminal 300 may be, for example, a general-purpose computer such as a personal computer. The game play terminal 300 comprises a processor 30, memory 31, storage 32, a communication IF 33, and an input / output IF 34. These components of the game play terminal 300 are electrically connected to each other via a communication bus.

[0036] As shown in FIG. 4, the game play terminal 300 according to this embodiment is, for example, included in an HMD (Head Mounted Display) set 1000. In other words, it can be expressed that the HMD set 1000 is included in the system 1, and it can also be expressed that the player plays the game using the HMD set 1000. Note that the device on which the player plays the game is not limited to the HMD set 1000. For example, the device may be any device that can allow the player to virtually experience the game. The device may also be realized as a smartphone, feature phone, tablet computer, laptop computer (so-called notebook computer), desktop computer, or the like. The device may also be a gaming device suitable for game play.

[0037] In addition to the game play terminal 300, the HMD set 1000 includes an HMD 500, an HMD sensor 510, a motion sensor 520, a display 530, and a controller 540. The HMD 500 includes a monitor 51, a gaze sensor 52, a first camera 53, a second camera 54, a microphone 55, and a speaker 56. The controller 540 may include the motion sensor 520.

[0038] The HMD 500 is worn on the player's head and can provide the player with a virtual space during operation. More specifically, the HMD 500 displays an image for the right eye and an image for the left eye on a monitor 51. When each eye of the player views the respective image, the player can recognize the image as a three-dimensional image based on the parallax between the two eyes. The HMD 500 can include both a so-called head-mounted display equipped with a monitor and a head-mounted device to which a smartphone or other terminal equipped with a monitor can be attached.

[0039] The monitor 51 is realized, for example, as a non-transparent display device. In one aspect, the monitor 51 is disposed on the main body of the HMD 500 so as to be positioned in front of both eyes of the player. Therefore, when the player views the three-dimensional image displayed on the monitor 51, the player can be immersed in the virtual space. In one aspect, the virtual space includes, for example, images of a background, objects that the player can control, and menus that the player can select. In another aspect, the monitor 51 can be realized as a liquid crystal monitor or an organic EL (Electro Luminescence) monitor provided in a so-called smartphone or other information display terminal.

[0040] In another aspect, the monitor 51 may be realized as a transmissive display device. In this case, the HMD 500 may be an open type such as glasses, rather than a closed type that covers the player's eyes as shown in FIG. 1 . The transmissive monitor 51 may be temporarily configured as a non-transmissive display device by adjusting its transmittance. The monitor 51 may be configured to simultaneously display a portion of an image constituting a virtual space and the real space. For example, the monitor 51 may display an image of the real space captured by a camera mounted on the HMD 500, or the transmittance of a portion of the monitor may be set high to make the real space visible.

[0041] In one aspect, the monitor 51 may include a sub-monitor for displaying an image for the right eye and a sub-monitor for displaying an image for the left eye. In another aspect, the monitor 51 may be configured to display an image for the right eye and an image for the left eye as a single image. In this case, the monitor 51 includes a high-speed shutter. The high-speed shutter operates to alternately display an image for the right eye and an image for the left eye so that the image is recognized by only one of the eyes.

[0042] In one aspect, the HMD 500 includes a plurality of light sources (not shown). Each light source is realized, for example, by an LED (Light Emitting Diode) that emits infrared rays. The HMD sensor 510 has a position tracking function for detecting the movement of the HMD 500. More specifically, the HMD sensor 510 reads a plurality of infrared rays emitted by the HMD 500 and detects the position and tilt of the HMD 500 in real space.

[0043] In another aspect, the HMD sensor 510 may be realized by a camera. In this case, the HMD sensor 510 can detect the position and tilt of the HMD 500 by performing image analysis processing using image information of the HMD 500 output from the camera.

[0044] In another aspect, the HMD 500 may include a sensor (not shown) as a position detector instead of or in addition to the HMD sensor 510. The HMD 500 may use the sensor to detect the position and tilt of the HMD 500 itself. For example, if the sensor is an angular velocity sensor, a geomagnetic sensor, or an acceleration sensor, the HMD 500 may use any of these sensors instead of the HMD sensor 510 to detect the position and tilt of the HMD 500 itself. As an example, if the sensor included in the HMD 500 is an angular velocity sensor, the angular velocity sensor detects the angular velocity of the HMD 500 around three axes in real space over time. The HMD 500 calculates changes over time in the angles of the HMD 500 around the three axes based on the angular velocities, and further calculates the tilt of the HMD 500 based on the changes over time in the angles.

[0045] The gaze sensor 52 detects the direction in which the player's right and left eyes are looking. In other words, the gaze sensor 52 detects the player's gaze. Detection of the gaze direction is achieved, for example, by a known eye tracking function. The gaze sensor 52 is achieved by a sensor having the eye tracking function. In some aspects, the gaze sensor 52 preferably includes a sensor for the right eye and a sensor for the left eye. The gaze sensor 52 may be, for example, a sensor that irradiates the player's right and left eyes with infrared light and detects the rotation angle of each eyeball by receiving light reflected from the cornea and iris of the irradiated light. The gaze sensor 52 can detect the player's gaze based on the detected rotation angles.

[0046] The first camera 53 captures the lower part of the player's face. More specifically, the first camera 53 captures the player's nose, mouth, etc. The second camera 54 captures the player's eyes, eyebrows, etc. The housing of the HMD 500 on the player's side is defined as the inside of the HMD 500, and the housing of the HMD 500 on the opposite side from the player is defined as the outside of the HMD 500. In one aspect, the first camera 53 may be located outside the HMD 500, and the second camera 54 may be located inside the HMD 500. Images generated by the first camera 53 and the second camera 54 are input to the game play terminal 300. In another aspect, the first camera 53 and the second camera 54 may be implemented as a single camera, and the player's face may be captured by this single camera.

[0047] The microphone 55 converts the player's speech into an audio signal (electrical signal) and outputs it to the game play terminal 300. The speaker 56 converts the audio signal into sound and outputs it to the player. In another aspect, the HMD 500 may include earphones instead of the speaker 56.

[0048] The controller 540 is connected to the game play terminal 300 by wire or wirelessly. The controller 540 accepts commands input from the player to the game play terminal 300. In one aspect, the controller 540 is configured to be held by the player. In another aspect, the controller 540 is configured to be attachable to the player's body or clothing. In yet another aspect, the controller 540 may be configured to output at least one of vibration, sound, and light based on a signal transmitted from the game play terminal 300. In yet another aspect, the controller 540 accepts operations from the player to control the position and movement of an object placed in a virtual space.

[0049] In one aspect, the controller 540 includes a plurality of light sources. Each light source is realized, for example, by an LED that emits infrared light. The HMD sensor 510 has a position tracking function. In this case, the HMD sensor 510 reads a plurality of infrared rays emitted by the controller 540 and detects the position and tilt of the controller 540 in real space. In another aspect, the HMD sensor 510 may be realized by a camera. In this case, the HMD sensor 510 can detect the position and tilt of the controller 540 by performing image analysis processing using image information of the controller 540 output from the camera.

[0050] In one aspect, the motion sensor 520 is attached to the player's hand and detects the player's hand movements. For example, the motion sensor 520 detects the rotation speed, number of rotations, etc. of the hand. The detected signal is sent to the game play terminal 300. The motion sensor 520 is provided, for example, in the controller 540. In one aspect, the motion sensor 520 is provided, for example, in the controller 540 configured to be held by the player. In another aspect, for safety in real space, the controller 540 is attached to a device, such as a glove, that is worn on the player's hand and does not easily fly away. In yet another aspect, a sensor not worn by the player may detect the player's hand movements. For example, a signal from a camera capturing an image of the player may be input to the game play terminal 300 as a signal representing the player's movements. The motion sensor 520 and the game play terminal 300 are connected to each other wirelessly, for example. In the case of wireless communication, the communication method is not particularly limited, and for example, Bluetooth or other known communication methods may be used.

[0051] The display 530 displays an image similar to the image displayed on the monitor 51. This allows users other than the player wearing the HMD 500 to view the same image as the player. The image displayed on the display 530 does not need to be a three-dimensional image, and may be an image for the right eye or an image for the left eye. Examples of the display 530 include a liquid crystal display and an organic EL monitor.

[0052] The game play terminal 300 controls the character controlled by the player to move and progress through the game based on various information acquired from the various components of the HMD 500, the controller 540, and the motion sensor 520. Here, "movement" includes moving various body parts, changing posture, changing facial expressions, moving, speaking, touching or moving objects placed in the virtual space, and using weapons, tools, etc. held by the character. That is, in this game, when the player moves various body parts, the character also moves various body parts in the same way as the player. Also, in this game, the character speaks what the player speaks. In other words, in this game, the character is an avatar object that acts as the player's alter ego. As an example, at least some of the character's movements may be performed by input from the player to the controller 540.

[0053] In this embodiment, the motion sensors 520 are attached to the player's hands, both feet, waist, and head, as an example. The motion sensors 520 attached to the player's hands may be provided in the controller 540, as described above. The motion sensor 520 attached to the player's head may be provided in the HMD 500. The motion sensors 520 may also be attached to the user's elbows and knees. By increasing the number of motion sensors 520 attached to the player, the player's movements can be more accurately reflected on the character. Furthermore, instead of attaching the motion sensors 520 to various parts of the body, the player may wear a suit to which one or more motion sensors 520 are attached. In other words, the method of motion capture is not limited to the example using the motion sensors 520.

[0054] (Distribution terminal 400) The distribution terminal 400 may be a mobile terminal such as a smartphone, a PDA (Personal Digital Assistant), or a tablet computer, or may be a so-called stationary terminal such as a desktop personal computer.

[0055] 5, the distribution terminal 400 includes a processor 40, a memory 41, a storage 42, a communication IF 43, an input / output IF 44, and a touch screen 45. Instead of or in addition to the touch screen 45, the distribution terminal 400 may include the input / output IF 44 to which a display (display unit) configured separately from the distribution terminal 400 main body can be connected.

[0056] The controller 1021 may have a physical input mechanism such as one or more buttons, levers, sticks, wheels, etc. The controller 1021 transmits to the streaming terminal 400 an output value based on an input operation input to the input mechanism by an operator of the streaming terminal 400 (a player in this embodiment). The controller 1021 may also have various sensors such as an acceleration sensor and an angular velocity sensor, and may transmit output values ​​of the various sensors to the streaming terminal 400. The above-mentioned output values ​​are received by the streaming terminal 400 via the communication IF 43.

[0057] The broadcasting terminal 400 may include a camera and a distance measurement sensor (both not shown). Instead of or in addition to the broadcasting terminal 400 including the camera and the distance measurement sensor, the controller 1021 may include the camera and the distance measurement sensor.

[0058] As described above, the streaming terminal 400 includes the communication IF 43, the input / output IF 44, and the touch screen 45 as an example of a mechanism for inputting information to the streaming terminal 400. Each of the above-described units serving as an input mechanism can be considered as an operation unit configured to accept input operations by the user.

[0059] If the operation unit is configured with touch screen 45, distribution terminal 400 identifies and accepts a user operation performed on input unit 451 of touch screen 45 as the user's input operation. Alternatively, if the operation unit is configured with communication IF 43, distribution terminal 400 identifies and accepts a signal (e.g., an output value) transmitted from controller 1021 as the user's input operation. Alternatively, if the operation unit is configured with input / output IF 44, distribution terminal 400 identifies and accepts a signal output from an input device (not shown) connected to input / output IF 44 as the user's input operation.

[0060] <Hardware components of each device> Processors 10, 20, 30, and 40 control the overall operation of user terminal 100, server 200, game play terminal 300, and distribution terminal 400, respectively. Processors 10, 20, 30, and 40 include a CPU (Central Processing Unit), an MPU (Micro Processing Unit), and a GPU (Graphics Processing Unit). Processors 10, 20, 30, and 40 read programs from storages 12, 22, 32, and 42, respectively, which will be described later. Processors 10, 20, 30, and 40 then load the read programs into memories 11, 21, 31, and 41, which will be described later, respectively. Processors 10, 20, and 30 execute the loaded programs.

[0061] Memories 11, 21, 31, and 41 are main storage devices. Memories 11, 21, 31, and 41 are composed of storage devices such as ROM (Read Only Memory) and RAM (Random Access Memory). Memory 11 provides a working area for processor 10 by temporarily storing programs and various data that processor 10 reads from storage 12 (described later). Memory 11 also temporarily stores various data generated while processor 10 is operating according to a program. Memory 21 provides a working area for processor 20 by temporarily storing various programs and data that processor 20 reads from storage 22 (described later). Memory 21 also temporarily stores various data generated while processor 20 is operating according to a program. Memory 31 provides a working area for processor 30 by temporarily storing various programs and data that processor 30 reads from storage 32 (described later). Memory 31 also temporarily stores various data generated while processor 30 is operating according to a program. The memory 41 temporarily stores programs and various data that the processor 40 reads from the storage 42 (described later), thereby providing a working area for the processor 40. The memory 41 also temporarily stores various data that the processor 40 generates while operating according to the programs.

[0062] In this embodiment, the program executed by processors 10 and 30 may be a game program for the game. In this embodiment, the program executed by processor 40 may be a distribution program for distributing action instruction data. Processor 10 may also execute a viewing program for playing video.

[0063] In this embodiment, the program executed by processor 20 may be at least one of the above-mentioned game program, distribution program, and viewing program. Processor 20 executes at least one of the game program, distribution program, and viewing program in response to a request from at least one of user terminal 100, game play terminal 300, and distribution terminal 400. The distribution program and the viewing program may be executed in parallel.

[0064] That is, the game program may be a program that realizes a game through cooperation between the user terminal 100, the server 200, and the game play terminal 300. The distribution program may be a program that realizes distribution of action instruction data through cooperation between the server 200 and the distribution terminal 400. The viewing program may be a program that realizes playback of a video through cooperation between the user terminal 100 and the server 200.

[0065] Storages 12, 22, 32, and 42 are auxiliary storage devices. Storages 12, 22, 32, and 42 are configured with storage devices such as flash memory or HDDs (Hard Disk Drives). Storages 12 and 32 store, for example, various data related to games. Storage 42 stores various data related to the distribution of action instruction data. Storage 12 stores various data related to the playback of moving images. Storage 22 may store at least some of the various data related to games, the distribution of action instruction data, and the playback of moving images.

[0066] Communication IFs 13, 23, 33, and 43 respectively control the transmission and reception of various types of data between user terminal 100, server 200, game play terminal 300, and distribution terminal 400. Communication IFs 13, 23, 33, and 43 control, for example, communication via a wireless LAN (Local Area Network), internet communication via a wired LAN, wireless LAN, or mobile phone network, and communication using short-range wireless communication, etc.

[0067] The input / output IFs 14, 24, 34, and 44 are interfaces through which the user terminal 100, the server 200, the game play terminal 300, and the distribution terminal 400 receive data input and output data, respectively. The input / output IFs 14, 24, 34, and 44 may input and output data via a USB (Universal Serial Bus) or the like. The input / output IFs 14, 24, 34, and 44 may include physical buttons, a camera, a microphone, a speaker, a mouse, a keyboard, a display, a stick, a lever, and the like. The input / output IFs 14, 24, 34, and 44 may also include a connection section for transmitting and receiving data to and from peripheral devices.

[0068] The touch screen 15 is an electronic component that combines an input unit 151 and a display unit 152 (display). The touch screen 45 is an electronic component that combines an input unit 451 and a display unit 452. The input units 151 and 451 are, for example, touch-sensitive devices, and are configured, for example, by touchpads. The display units 152 and 452 are, for example, liquid crystal displays, organic EL (Electro-Luminescence) displays, or the like.

[0069] The input units 151 and 451 have a function of detecting the position where a user operation (mainly a physical contact operation such as a touch operation, a slide operation, a swipe operation, a pinch-in / pinch-out operation, or a tap operation) is input on the input surface, and transmitting information indicating the position as an input signal. The input units 151 and 451 may be provided with a touch sensing unit (not shown). The touch sensing unit may be of any type, such as a capacitive type or a resistive type.

[0070] Although not shown, the user terminal 100 and the streaming terminal 400 may each include one or more sensors for identifying the holding posture of the user terminal 100 and the streaming terminal 400. The sensors may be, for example, an acceleration sensor or an angular velocity sensor.

[0071] If the user terminal 100 and the streaming terminal 400 are equipped with sensors, the processors 10 and 40 can identify the holding orientation of the user terminal 100 and the streaming terminal 400, respectively, from the sensor output and perform processing according to the holding orientation. For example, when the user terminal 100 and the streaming terminal 400 are held in portrait orientation, the processors 10 and 40 can set the display units 152 and 452 to display a vertically elongated image in portrait screen mode. On the other hand, when the user terminal 100 and the streaming terminal 400 are held in landscape orientation, the processors 10 and 40 can set the display units 152 and 452 to display a horizontally elongated image in landscape screen mode. In this way, the processors 10 and 40 can switch between portrait screen mode and landscape screen mode depending on the holding orientation of the user terminal 100 and the streaming terminal 400, respectively.

[0072] <Functional configuration of System 1> 6 is a block diagram showing the functional configuration of the user terminal 100, the server 200, and the HMD set 1000 included in the system 1. FIG. 7 is a block diagram showing the functional configuration of the distribution terminal 400 shown in FIG.

[0073] The user terminal 100 functions as an input device that accepts user input operations and as an output device that outputs game images and sounds. The user terminal 100 functions as a control unit 110 and a storage unit 120 through cooperation of a processor 10, a memory 11, a storage 12, a communication IF 13, an input / output IF 14, a touch screen 15, and the like.

[0074] The server 200 has a function of mediating the transmission and reception of various types of information among the user terminal 100, the HMD set 1000, and the distribution terminal 400. The server 200 functions as a control unit 210 and a storage unit 220 through cooperation of the processor 20, the memory 21, the storage 22, the communication IF 23, the input / output IF 24, and the like.

[0075] The HMD set 1000 (game playing terminal 300) functions as an input device that accepts input operations from the player, as an output device that outputs game images and sounds, and as a function that transmits game progress information to the user terminal 100 in real time via the server 200. The HMD set 1000 functions as a control unit 310 and a memory unit 320 through cooperation of the processor 30, memory 31, storage 32, communication IF 33, input / output IF 34 of the game playing terminal 300, as well as the HMD 500, HMD sensor 510, motion sensor 520, and controller 540.

[0076] The distribution terminal 400 has a function of generating action instruction data and transmitting the action instruction data to the user terminal 100 via the server 200. The distribution terminal 400 functions as a control unit 410 and a storage unit 420 through cooperation of the processor 40, memory 41, storage 42, communication IF 43, input / output IF 44, touch screen 45, etc.

[0077] (Data stored in the memory unit of each device) The storage unit 120 stores a game program 131 (program), game information 132, and user information 133. The storage unit 220 stores a game program 231, game information 232, user information 233, and a user list 234. The storage unit 320 stores a game program 331, game information 332, and user information 333. The storage unit 420 stores a user list 421, a motion list 422, and a distribution program 423 (program, second program).

[0078] The game programs 131, 231, and 331 are game programs executed by the user terminal 100, the server 200, and the HMD set 1000, respectively. The game is realized by the cooperative operation of each device based on the game programs 131, 231, and 331. The game programs 131 and 331 may be stored in the storage unit 220 and downloaded to the user terminal 100 and the HMD set 1000, respectively. In this embodiment, the user terminal 100 renders data received from the distribution terminal 400 based on the game program 131 and plays videos. In other words, the game program 131 is also a program for playing videos using video instruction data distributed from the distribution terminal 400. The program for playing the videos may be different from the game program 131. In this case, the storage unit 120 stores the program for playing the videos separately from the game program 131.

[0079] The game information 132, 232, and 332 are data that are referenced by the user terminal 100, the server 200, and the HMD set 1000 when executing a game program, respectively. The user information 133, 233, and 333 are data related to the account of the user of the user terminal 100. The game information 232 is the game information 132 of each user terminal 100 and the game information 332 of the HMD set 1000. The user information 233 is the user information 133 of each user terminal 100 and the user information of the player included in the user information 333. The user information 333 is the user information 133 of each user terminal 100 and the user information of the player.

[0080] The user list 234 and the user list 421 are lists of users who have participated in the game. The user list 234 and the user list 421 may include a list of users who participated in the player's most recent game play, as well as a list of users who participated in each game play prior to that game play. The motion list 422 is a list of multiple motion data created in advance. The motion list 422 is, for example, a list in which information identifying each motion (for example, a motion name) is associated with each piece of motion data. The distribution program 423 is a program for realizing the distribution of action instruction data for playing a video on the user terminal 100 to the user terminal 100.

[0081] (Functional configuration of server 200) The control unit 210 executes the game program 231 stored in the storage unit 220 to comprehensively control the server 200. For example, the control unit 210 mediates the transmission and reception of various types of information among the user terminal 100, the HMD set 1000, and the distribution terminal 400.

[0082] The control unit 210 functions as a communication intermediation unit 211, a log generation unit 212, and a list generation unit 213 in accordance with the description of the game program 231. The control unit 210 can also function as other functional blocks (not shown) to mediate the sending and receiving of various information related to game play and distribution of action instruction data, and to support the progress of the game.

[0083] The communication intermediation unit 211 mediates the transmission and reception of various types of information among the user terminal 100, the HMD set 1000, and the distribution terminal 400. For example, the communication intermediation unit 211 transmits game progress information received from the HMD set 1000 to the user terminal 100. The game progress information includes data indicating the movement of a character controlled by a player, the character's parameters, items and weapons held by the character, enemy characters, and other information. The server 200 transmits the game progress information to the user terminals 100 of all users participating in the game. In other words, the server 200 transmits common game progress information to the user terminals 100 of all users participating in the game. As a result, the game progresses in the same manner as in the HMD set 1000 in each of the user terminals 100 of all users participating in the game.

[0084] Furthermore, for example, the communication intermediation unit 211 transmits information received from one of the user terminals 100 to assist the player in progressing through the game to the other user terminals 100 and the HMD set 1000. As an example, the information may be item information indicating an item that allows the player to progress through the game advantageously and is provided to the player (character). The item information includes information indicating the user who provided the item (user name, user ID, etc.). The communication intermediation unit 211 may also mediate the distribution of action instruction data from the distribution terminal 400 to the user terminal 100.

[0085] The log generation unit 212 generates a game progress log based on the game progress information received from the HMD set 1000. The list generation unit 213 generates a user list 234 after game play ends. As will be described in detail later, each user in the user list 234 is associated with a tag indicating the content of the support provided by that user to the player. The list generation unit 213 generates a tag based on the game progress log generated by the log generation unit 212 and associates it with the corresponding user. Note that the list generation unit 213 may associate the content of the support provided by each user to the player, which is input by a game operator or the like using a terminal device such as a personal computer, with the corresponding user as a tag. This provides more detailed information about the support provided by each user. Note that when a user participates in a game, the user terminal 100 transmits information identifying the user to the server 200 based on the user's operation. For example, the user terminal 100 transmits the user ID input by the user to the server 200. In other words, the server 200 holds information identifying each user for all users participating in the game. The list generation unit 213 may generate the user list 234 using this information.

[0086] (Functional configuration of HMD set 1000) The control unit 310 executes a game program 331 stored in the storage unit 320 to exercise overall control over the HMD set 1000. For example, the control unit 310 progresses the game in accordance with the game program 331 and the player's operations. Furthermore, while the game is progressing, the control unit 310 communicates with the server 200 as necessary to send and receive information. The control unit 310 may also send and receive information directly to and from the user terminal 100 without going through the server 200.

[0087] The control unit 310 functions as an operation reception unit 311, a display control unit 312, a UI control unit 313, an animation generation unit 314, a game progression unit 315, a virtual space control unit 316, and a reaction processing unit 317 in accordance with the description of the game program 331. The control unit 310 can also function as other functional blocks (not shown) to control characters appearing in the game in accordance with the nature of the game being executed.

[0088] The operation reception unit 311 detects and receives input operations from the player. The operation reception unit 311 receives signals input from the HMD 500, the motion sensor 520, the controller 540, etc., determines what kind of input operation has been performed, and outputs the results to each element of the control unit 310.

[0089] The UI control unit 313 controls user interface (hereinafter referred to as UI) images to be displayed on the monitor 51, the display 530, etc. The UI images are tools that allow the player to input necessary for the progress of the game to the HMD set 1000, or to obtain information output during the progress of the game from the HMD set 1000. The UI images are not limited to these, but include, for example, icons, buttons, lists, menu screens, etc.

[0090] The animation generation unit 314 generates animations showing the motion of various objects based on the control modes of the various objects. For example, the animation generation unit 314 may generate animations that express the movement of an object (e.g., a player's avatar object) as if it were actually there, moving its mouth, changing its facial expression, etc.

[0091] The game progression unit 315 progresses the game based on the game program 331, input operations by the player, and actions of the avatar object in response to the input operations. For example, the game progression unit 315 performs predetermined game processing when the avatar object performs a predetermined action. Furthermore, for example, the game progression unit 315 may receive information representing user operations on the user terminal 100 and perform game processing based on the user operations. Furthermore, the game progression unit 315 generates game progression information in accordance with the progress of the game and transmits it to the server 200. The game progress information is transmitted to the user terminal 100 via the server 200. In this way, the progress of the game on the HMD set 1000 is shared with the user terminal 100. In other words, the progress of the game on the HMD set 1000 and the progress of the game on the user terminal 100 are synchronized.

[0092] The virtual space control unit 316 performs various controls related to the virtual space provided to the player in accordance with the progress of the game. As an example, the virtual space control unit 316 generates various objects and places them in the virtual space. The virtual space control unit 316 also places a virtual camera in the virtual space. The virtual space control unit 316 also operates the various objects placed in the virtual space in accordance with the progress of the game. The virtual space control unit 316 also controls the position and tilt of the virtual camera placed in the virtual space in accordance with the progress of the game.

[0093] The display control unit 312 outputs a game screen reflecting the results of the processing executed by the above-mentioned elements to the monitor 51 and the display 530. The display control unit 312 may display an image based on the field of view from a virtual camera placed in virtual space as a game screen on the monitor 51 and the display 530. The display control unit 312 may also include animation generated by the animation generation unit 314 in the game screen. The display control unit 312 may also draw the above-mentioned UI image controlled by the UI control unit 313 so as to be superimposed on the game screen.

[0094] The reaction processing unit 317 receives feedback from the user of the user terminal 100 regarding the reaction of the player to the game play and outputs this to the player. In this embodiment, for example, the user terminal 100 can create a comment (message) addressed to an avatar object based on an input operation by the user. The reaction processing unit 317 receives comment data of the comment and outputs it. The reaction processing unit 317 may display text data corresponding to the user's comment on the monitor 51 or the display 530, or may output audio data corresponding to the user's comment from a speaker (not shown). In the former case, the reaction processing unit 317 may draw an image corresponding to the text data (i.e., an image including the content of the comment) by superimposing it on the game screen.

[0095] (Functional configuration of user terminal 100) The control unit 110 executes a game program 131 stored in the storage unit 120 to exercise overall control over the user terminal 100. For example, the control unit 110 progresses the game in accordance with the game program 131 and user operations. Furthermore, while the game is progressing, the control unit 110 communicates with the server 200 as necessary to send and receive information. The control unit 110 may also send and receive information directly to the HMD set 1000 without going through the server 200.

[0096] The control unit 110 functions as an operation reception unit 111, a display control unit 112, a UI control unit 113, an animation generation unit 114, a game progression unit 115, a virtual space control unit 116, and a video playback unit 117 in accordance with the description of the game program 131. The control unit 110 can also function as other functional blocks (not shown) for the progression of the game in accordance with the nature of the game being executed.

[0097] The operation reception unit 111 detects and receives an input operation by the user to the input unit 151. The operation reception unit 111 determines what kind of input operation has been performed from the action the user has taken on the touch screen 15 and the console via the other input / output IF 14, and outputs the result to each element of the control unit 110.

[0098] For example, the operation reception unit 111 receives an input operation on the input unit 151, detects the coordinates of the input position of the input operation, and identifies the type of the input operation. The operation reception unit 111 identifies, as the type of input operation, for example, a touch operation, a slide operation, a swipe operation, a pinch-in / pinch-out operation, a tap operation, etc. Furthermore, when a continuously detected input is interrupted, the operation reception unit 111 detects that the touch input from the touch screen 15 has been released.

[0099] The UI control unit 113 controls UI images to be displayed on the display unit 152 to construct a UI in response to at least one of a user's input operation and received game progress information. The UI images are tools that allow the user to input necessary information for the progress of the game to the user terminal 100, or tools that allow the user to obtain information output during the progress of the game from the user terminal 100. The UI images are not limited to these, but include, for example, icons, buttons, lists, menu screens, etc.

[0100] The animation generation unit 114 generates animations showing the motions of various objects based on the control modes of the various objects.

[0101] The game progression unit 115 progresses the game based on the game program 131, the received game progress information, and input operations by the user. When a predetermined game process is performed in response to an input operation by the user, the game progression unit 115 transmits information related to the game process to the HMD set 1000 via the server 200. This causes the predetermined game process to be shared in the HMD set 1000. In other words, the progress of the game in the HMD set 1000 and the progress of the game in the user terminal 100 are synchronized. The predetermined game process is, for example, a process of providing an item to an avatar object, and in this example, the information related to the game process is the item information described above.

[0102] The virtual space control unit 116 performs various controls related to the virtual space provided to the user in accordance with the progress of the game. As an example, the virtual space control unit 116 generates various objects and places them in the virtual space. The virtual space control unit 116 also places a virtual camera in the virtual space. The virtual space control unit 116 also operates the various objects placed in the virtual space in accordance with the progress of the game, specifically, the received game progress information. The virtual space control unit 316 also controls the position and tilt of the virtual camera placed in the virtual space in accordance with the progress of the game, specifically, the received game progress information.

[0103] The display control unit 112 outputs to the display unit 152 a game screen that reflects the results of the processing executed by each of the above-mentioned elements. The display control unit 112 may display, as a game screen, an image based on a field of view from a virtual camera placed in a virtual space provided to the user on the display unit 152. The display control unit 112 may also include animation generated by the animation generation unit 114 in the game screen. The display control unit 112 may also draw the above-mentioned UI image controlled by the UI control unit 113, superimposed on the game screen. In any case, the game screen displayed on the display unit 152 is the same game screen as the game screen displayed on other user terminals 100 and the HMD set 1000.

[0104] The video playback unit 117 analyzes (renders) the action instruction data received from the distribution terminal 400 and plays back the video.

[0105] (Functional configuration of distribution terminal 400) The control unit 410 executes a program (not shown) stored in the storage unit 420 to exercise overall control over the distribution terminal 400. For example, the control unit 410 generates action instruction data in accordance with the program and operations by the user (player in this embodiment) of the distribution terminal 400, and distributes the data to the user terminal 100. The control unit 410 also communicates with the server 200 as necessary to send and receive information. The control unit 410 may also send and receive information directly to and from the user terminal 100 without going through the server 200.

[0106] According to the description of the program, the control unit 410 functions as a communication control unit 411, a display control unit 412, an operation reception unit 413, a voice reception unit 414, a motion identification unit 415, and an action instruction data generation unit 416. The control unit 410 can also function as other functional blocks (not shown) for generating and distributing action instruction data.

[0107] The communication control unit 411 controls transmission and reception of information to and from the server 200 or the user terminal 100 via the server 200. As an example, the communication control unit 411 receives a user list 421 from the server 200. In addition, as an example, the communication control unit 411 transmits operation instruction data to the user terminal 100.

[0108] The display control unit 412 outputs various screens reflecting the results of the processing performed by each element to the display unit 452. As an example, the display control unit 412 displays a screen including the received user list 234. Also, as an example, the display control unit 412 displays a screen including a motion list 422 for allowing the player to select motion data for moving the avatar object, which is included in the action instruction data to be distributed.

[0109] The operation reception unit 413 detects and receives input operations by the player via the input unit 151. The operation reception unit 111 determines what kind of input operation has been performed from the action performed by the player on the touch screen 45 and the console via other input / output IF 44, and outputs the result to each element of the control unit 410.

[0110] For example, the operation reception unit 413 receives an input operation on the input unit 451, detects the coordinates of the input position of the input operation, and identifies the type of the input operation. The operation reception unit 413 identifies, as the type of input operation, for example, a touch operation, a slide operation, a swipe operation, a pinch-in / pinch-out operation, a tap operation, etc. Furthermore, when a continuously detected input is interrupted, the operation reception unit 413 detects that the touch input from the touch screen 45 has been released.

[0111] The voice receiving unit 414 receives voice generated around the distribution terminal 400 and generates voice data of the voice. As an example, the voice receiving unit 414 receives voice uttered by a player and generates voice data of the voice.

[0112] The motion specifying unit 415 specifies the motion data selected by the player from the motion list 422 in response to the input operation by the player.

[0113] The action instruction data generating unit 416 generates action instruction data. As an example, the action instruction data generating unit 416 generates action instruction data including the generated voice data and the identified motion data.

[0114] Note that the functions of the HMD set 1000, the server 200, and the user terminal 100 shown in FIG. 6 and the functions of the streaming terminal 400 shown in FIG. 7 are merely examples. Each of the HMD set 1000, the server 200, the user terminal 100, and the streaming terminal 400 may have at least some of the functions of the other devices. Furthermore, a device other than the HMD set 1000, the server 200, the user terminal 100, and the streaming terminal 400 may be a component of the system 1, and the other device may execute some of the processing in the system 1. That is, in this embodiment, the computer that executes the game program may be any of the HMD set 1000, the server 200, the user terminal 100, the streaming terminal 400, and other devices, or may be realized by a combination of multiple devices.

[0115] <Virtual space control processing> FIG. 8 is a flowchart showing an example of the flow of control processing for a virtual space provided to a player and a virtual space provided to a user of user terminal 100. FIG. 9 is a diagram showing a virtual space 600A provided to a player and a field of view image visually recognized by the player according to an embodiment. FIG. 10 is a diagram showing a virtual space 600B provided to a user of user terminal 100 and a field of view image visually recognized by the user according to an embodiment. Hereinafter, when there is no need to distinguish between virtual spaces 600A and 600B, they will be referred to as "virtual space 600."

[0116] In step S1, processor 30, functioning as virtual space control unit 316, defines virtual space 600A shown in FIG. 9. Processor 30 defines virtual space 600A using virtual space data (not shown). The virtual space data may be stored in game play terminal 300, may be generated by processor 30 based on game program 331, or may be acquired by processor 30 from an external device such as server 200.

[0117] As an example, virtual space 600 has a spherical structure that covers the entire 360-degree area from a point defined as the center. 9 and 10 illustrate only the upper half of the celestial sphere in virtual space 600 in order to avoid complicating the explanation.

[0118] In step S2, processor 30, functioning as virtual space control unit 316, places avatar object 610 (character) in virtual space 600A. Avatar object 610 is an avatar object associated with a player, and moves in accordance with an input operation by the player.

[0119] In step S3, processor 30, functioning as virtual space control unit 316, places other objects in virtual space 600A. In the example of Fig. 9, processor 30 places objects 631 to 634. The other objects may include, for example, character objects (so-called non-player characters, NPCs) that operate according to game program 331, controllable objects such as virtual hands, and objects that resemble animals, plants, artificial objects, natural objects, etc. that are placed as the game progresses.

[0120] In step S4, processor 30, functioning as virtual space control unit 316, places virtual camera 620A in virtual space 600A. Processor 30 places virtual camera 620A at the position of the head of avatar object 610, for example.

[0121] In step S5, processor 30 displays field of view image 650 on monitor 51 and display 530. Processor 30 defines field of view area 640A, which is the field of view from virtual camera 620A in virtual space 600A, according to the initial position and tilt of virtual camera 620A. Processor 30 then defines field of view image 650 corresponding to field of view area 640A. Processor 30 outputs field of view image 650 to monitor 51 and display 530, thereby displaying field of view image 650 on HMD 500 and display 530.

[0122] In the example of FIG. 9, as shown in FIG. 9(A), a portion of object 634 is included in field of view area 640A, and therefore field of view image 650 includes a portion of object 634 as shown in FIG. 9(B).

[0123] In step S6, processor 30 transmits initial placement information to user terminal 100 via server 200. The initial placement information is information indicating the initial placement positions of various objects in virtual space 600A. In the example of FIG. 9, the initial placement information includes information on the initial placement positions of avatar object 610 and objects 631 to 634. The initial placement information can also be expressed as one piece of game progress information.

[0124] In step S7, processor 30, functioning as virtual space control unit 316, controls virtual camera 620A in accordance with the movement of HMD 500. Specifically, processor 30 controls the orientation and tilt of virtual camera 620A in accordance with the movement of HMD 500, i.e., the pose of the player's head. As will be described later, when the player moves his / her head (changes the pose of his / her head), processor 30 moves the head of avatar object 610 in accordance with this movement. Processor 30 controls the orientation and tilt of virtual camera 620A, for example, so that the line of sight of avatar object 610 and the line of sight of virtual camera 620A coincide with each other. In step S8, processor 30 updates field of view image 650 in accordance with the change in the orientation and tilt of virtual camera 620A.

[0125] In step S9, processor 30, functioning as virtual space control unit 316, moves avatar object 610 in accordance with the movement of the player. As an example, processor 30 moves avatar object 610 in virtual space 600A in accordance with the player's movement in real space. Furthermore, processor 30 moves the head of avatar object 610 in virtual space 600A in accordance with the player's movement of his / her head in real space.

[0126] In step S10, processor 30, functioning as virtual space control unit 316, moves virtual camera 620A to follow avatar object 610. That is, virtual camera 620A is always located at the head position of avatar object 610, even if avatar object 610 moves.

[0127] Processor 30 updates field of view image 650 in accordance with the movement of virtual camera 620A. That is, processor 30 updates field of view 640A in accordance with the pose of the player's head and the position of virtual camera 620A in virtual space 600A. As a result, field of view image 650 is updated.

[0128] In step S11, processor 30 transmits action instruction data for avatar object 610 to user terminal 100 via server 200. The action instruction data here includes at least any of motion data capturing the player's actions during the virtual experience (e.g., while playing a game), audio data of the player's voice, and operation data indicating the content of an input operation on controller 540. When the player is playing a game, the action instruction data is transmitted to user terminal 100 as, for example, game progress information.

[0129] The processing of steps S7 to S11 is continuously and repeatedly executed while the player is playing the game.

[0130] In step S21, the processor 10 of the user terminal 100 of the user 3 defines the virtual space 600B shown in FIG. 10 as the virtual space control unit 116. The processor 10 defines the virtual space 600B using virtual space data (not shown). The virtual space data may be stored in the user terminal 100, may be generated by the processor 10 based on the game program 131, or may be acquired by the processor 10 from an external device such as the server 200.

[0131] In step S22, processor 10 receives the initial placement information. In step S23, processor 10, functioning as virtual space control unit 116, places various objects in virtual space 600B in accordance with the initial placement information. In the example of Fig. 10, the various objects are avatar object 610 and objects 631 to 634.

[0132] In step S24, the processor 10, functioning as the virtual space control unit 116, places a virtual camera 620B in the virtual space 600B. As an example, the processor 10 places the virtual camera 620B at the position shown in FIG.

[0133] In step S25, processor 10 displays field of view image 660 on display unit 152. Processor 10 defines field of view area 640B, which is the field of view from virtual camera 620B in virtual space 600B, according to the initial position and tilt of virtual camera 620B. Processor 10 then defines field of view image 660 corresponding to field of view area 640B. Processor 10 outputs field of view image 660 to display unit 152, thereby displaying field of view image 660 on display unit 152.

[0134] In the example of Figure 10, as shown in Figure 10(A), avatar object 610 and object 631 are included in field of view area 640B, so field of view image 660 includes avatar object 610 and object 631 as shown in Figure 10(B).

[0135] In step S26, processor 10 receives the action instruction data. In step S27, processor 10, as virtual space control unit 116, moves avatar object 610 in virtual space 600B in accordance with the action instruction data. In other words, processor 10 plays back a video of avatar object 610 moving by real-time rendering.

[0136] In step S28, the processor 10, functioning as the virtual space control unit 116, controls the virtual camera 620B in response to a user operation received by the operation receiving unit 111. In step S29, the processor 10 updates the field of view image 660 in response to changes in the position of the virtual camera 620B in the virtual space 600B and the orientation and tilt of the virtual camera 620B. Note that in step S28, the processor 10 may automatically control the virtual camera 620B in response to a movement of the avatar object 610, such as a movement or change in orientation of the avatar object 610. For example, the processor 10 may automatically move the virtual camera 620B or change its orientation and tilt so that the avatar object 610 is always photographed from the front. As another example, the processor 10 may automatically move the virtual camera 620B or change its orientation and tilt so that the avatar object 610 is always photographed from behind in response to the movement of the avatar object 610.

[0137] In this way, in the virtual space 600A, the avatar object 610 moves in accordance with the player's movements. Action instruction data indicating this movement is transmitted to the user terminal 100. In the virtual space 600B, the avatar object 610 moves in accordance with the received action instruction data. As a result, the avatar object 610 performs the same movement in the virtual space 600A and the virtual space 600B. In other words, the user 3 can visually recognize the movement of the avatar object 610 in accordance with the player's movement using the user terminal 100.

[0138] <Game Overview> 11 is a diagram showing another example of a field of view image displayed on the user terminal 100. Specifically, it is a diagram showing an example of a game screen of the game (this game) executed by the system 1 and played by the player.

[0139] As an example, this game is a game in which an avatar object 610 that controls a weapon such as a gun or a knife and a plurality of enemy objects 671 that are NPCs appear in a virtual space 600, and the avatar object 610 fights against the enemy objects 671. Various game parameters such as the stamina of the avatar object 610, the number of usable magazines, the number of bullets remaining in the gun, and the number of remaining enemy objects 671 are updated as the game progresses.

[0140] The game includes multiple stages, and the player can clear each stage by fulfilling a predetermined achievement condition associated with the stage. The predetermined achievement condition may include, for example, defeating all enemy objects 671 that appear, defeating a boss object among the enemy objects 671 that appear, acquiring a predetermined item, or reaching a predetermined location. The achievement condition is defined in the game program 131. In the game, the player clears a stage when the achievement condition is fulfilled in accordance with the content of the game; in other words, the victory of the avatar object 610 over the enemy object 671 (the victory or defeat between the avatar object 610 and the enemy object 671) is determined. In contrast, for example, if the game executed by the system 1 is a racing game, the ranking of the avatar object 610 is determined when the condition of reaching the goal is fulfilled.

[0141] In this game, in order to share a virtual space between the HMD set 1000 and multiple user terminals 100, game progress information is live-streamed to multiple user terminals 100 at predetermined time intervals. As a result, a field of view image of the field of view area defined by the virtual camera 620B corresponding to the user terminal 100 is displayed on the touch screen 15 of the user terminal 100 while the user is watching the game. In addition, parameter images representing the avatar object 610's vitality, the number of available magazines, the number of remaining bullets in the gun, the number of remaining enemy objects 671, etc. are superimposed and displayed in the upper right and upper left sections of the field of view image. This field of view image can also be referred to as a game screen.

[0142] As described above, the game progress information includes motion data capturing the player's actions, audio data of the voices uttered by the player, and operation data indicating the content of input operations on controller 540. These pieces of data are, in other words, information for specifying the position, posture, orientation, etc. of avatar object 610, information for specifying the position, posture, orientation, etc. of enemy object 671, and information for specifying the positions, etc. of other objects (e.g., obstacle objects 672, 673). Processor 10 analyzes (renders) the game progress information to specify the position, posture, orientation, etc. of each object.

[0143] The game information 132 includes data on various objects such as the avatar object 610, the enemy object 671, and the obstacle objects 672 and 673. The processor 10 uses this data and the analysis results of the game progress information to update the position, posture, orientation, and so on of each object. As a result, the game progresses, and each object in the virtual space 600B moves in the same way as each object in the virtual space 600A. Specifically, in the virtual space 600B, each object including the avatar object 610 moves based on the game progress information, regardless of whether or not a user operates the user terminal 100.

[0144] As an example, UI images 701 and 702 are displayed superimposed on the field of view image on the touch screen 15 of the user terminal 100. The UI image 701 is a UI image that accepts an operation for displaying, on the touch screen 15, a UI image 711 that accepts, from the user 3, an item insertion operation for supporting the avatar object 610. The UI image 702 is a UI image that accepts an operation for displaying, on the touch screen 15, a UI image (described later) that accepts, from the user 3, an operation for inputting and transmitting a comment for the avatar object 610 (in other words, the player 4). The operation accepted by the UI images 701 and 702 may be, for example, an operation of tapping the UI images 701 and 702.

[0145] When UI image 701 is tapped, UI image 711 is displayed superimposed on the field of view image. UI image 711 includes, for example, UI image 711A with a magazine icon, UI image 711B with a first aid kit icon, UI image 711C with a traffic cone icon, and UI image 711D with a barricade icon. An item insertion operation corresponds to, for example, tapping any of the UI images.

[0146] As an example, when UI image 711A is tapped, the number of remaining bullets in the gun used by the avatar object 610 increases. When UI image 711B is tapped, the stamina of the avatar object 610 is restored. When UI images 711C and 711D are tapped, obstacle objects 672 and 673 that hinder the movement of the enemy object 671 are placed in the virtual space. One of the obstacle objects 672 and 673 may hinder the movement of the enemy object 671 more than the other.

[0147] The processor 10 transmits item insertion information indicating that an item insertion operation has been performed to the server 200. The item insertion information includes at least information for identifying the type of item specified by the item insertion operation. The item insertion information may also include other information related to the item, such as information indicating the position where the item is to be placed. The item insertion information is transmitted to other user terminals 100 and the HMD set 1000 via the server 200.

[0148] 12 is a diagram showing another example of the field of view image displayed on the user terminal 100. Specifically, it is a diagram showing an example of the game screen of the present game, and is a diagram for explaining communication between the player and the user terminal 100 during game play.

[0149] 12(A), the user terminal 100 causes the avatar object 610 to make an utterance 691. Specifically, the user terminal 100 causes the avatar object 610 to make an utterance 691 in accordance with the voice data included in the game progress information. The content of the utterance 691 is "I'm out of bullets!" uttered by the player 4. In other words, the content of the utterance 691 informs each user that the magazine is empty and the number of bullets loaded in the gun is one, and therefore, the user is likely to lose the means to attack the enemy object 671.

[0150] 12(A) uses a speech bubble to visually indicate the speech of avatar object 610, but in reality, audio is output from the speaker of user terminal 100. Along with the audio output, the speech bubble shown in FIG. 12(A) (i.e., a speech bubble containing the text of the audio content) may be displayed in the field of view image. This also applies to utterance 692, which will be described later.

[0151] When a tap operation on the UI image 702 is received, the user terminal 100 displays UI images 705 and 706 (message UIs) superimposed on the field of view image, as shown in Fig. 12(B). The UI image 705 is a UI image that displays a comment for the avatar object 610 (in other words, the player). The UI image 706 is a UI image that receives a comment sending operation from the user 3 to send the input comment.

[0152] For example, when the user terminal 100 receives a tap operation on the UI image 705, the user terminal 100 displays a UI image resembling a keyboard (not shown, hereinafter simply referred to as "keyboard") on the touch screen 15. The user terminal 100 displays text corresponding to the user's input operation on the keyboard on the UI image 705. In the example of FIG. 12(B), the text "I'll send you a magazine" is displayed on the UI image 705.

[0153] After inputting text, for example, when the user terminal 100 receives a tap operation on the UI image 706, the user terminal 100 transmits comment information including information indicating the input content (content of the text) and information indicating the user to the server 200. The comment information is transmitted to other user terminals 100 and the HMD set 1000 via the server 200.

[0154] UI image 703A is a UI image showing the username of the user who sent the comment, and UI image 704A is a UI image showing the content of the comment sent by that user. In the example of FIG. 12(B), a user with the username "BBBBB" sent comment information saying "Danger!" using his / her user terminal 100, and as a result, UI image 703A and UI image 704A are displayed. UI image 703A and UI image 704A are displayed on the touch screens 15 of all user terminals 100 participating in the game and on the monitor 51 of the HMD 500. Note that UI images 703A and 704A may be a single UI image. In other words, a single UI image may include the username and the content of the comment.

[0155] In the example of Fig. 12(C), a user with the user name "AAAAA", who is a user of the user terminal 100 shown in Fig. 12, has input and sent a comment as described above, and as a result, UI images 703B and 704B are displayed on the touch screen 15. UI image 703B includes the user name "AAAAA", and UI image 704B includes the comment "I'll send you a magazine!" that was input in the example of Fig. 12(B).

[0156] 12(C) is a view image 611 after user "AAAAAA" further inputs a tap operation on UI image 701, causes UI image 711 to be displayed on touch screen 15, and inputs a tap operation on UI image 711A. That is, as a result of item insertion information indicating a magazine being transmitted from user terminal 100 of user "AAAAAA" to other user terminals 100 and HMD set 1000, user terminal 100 and HMD set 1000 place an effect object 674 (described below) in virtual space 600. As an example, after the elapsed time indicated in the item insertion information has elapsed, user terminal 100 and HMD set 1000 execute an effect related to effect object 674 and execute a process to activate the effect of the item object.

[0157] In the example of FIG. 12(D), the number of magazines increases from 0 to 1 by executing a process to activate the effect of the item object. As a result, the player utters "Thank you!" to the user "AAAAA", and the voice data of this utterance is transmitted to each user terminal 100. As a result, each user terminal 100 outputs the voice "Thank you!" as the utterance 692 of the avatar object 610.

[0158] As described above, in this game, communication between the users and the avatar object 610 is realized by the output of speech sounds from the avatar object 610 based on the speech of the player and the input of comments by each user.

[0159] (Game Progression Processing in Game Play Terminal 300) FIG. 13 is a flowchart showing an example of the flow of the game progression process executed by the game play terminal 300.

[0160] In step S31, the processor 30, as the game progression unit 315, progresses the game based on the game program 331 and the movements of the player. In step S32, the processor 30 generates game progress information and distributes it to the user terminals 100. Specifically, the processor 30 transmits the generated game progress information to each user terminal 100 via the server 200.

[0161] In step S33, upon receiving the item insertion information (YES in S33), in step S34, processor 30 places an item object in virtual space 600A based on the item insertion information. As an example, before placing the item object, processor 30 places an effect object 674 in virtual space 600A (see FIG. 11(C)). Effect object 674 may be, for example, an object resembling a gift box. As an example, processor 30 may execute an effect related to effect object 674 after the elapse of the time indicated in the item insertion information. The effect may be, for example, an animation in which the lid of the gift box opens. After the execution of the animation, processor 30 executes processing to activate the effect of the item object. For example, in the example of FIG. 11(D), obstacle object 673 is placed.

[0162] After the animation is executed, processor 30 may place an item object corresponding to the tapped UI image in virtual space 600A. For example, when a tap operation is performed on UI image 711A, processor 30 places a magazine object indicating a magazine in virtual space 600A after the animation is executed. Furthermore, when a tap operation is performed on UI image 711B, processor 30 places a first aid kit object indicating a first aid kit in virtual space 600A after the animation is executed. For example, when avatar object 610 moves to the position of the magazine object or the first aid kit object, processor 30 may execute processing to activate the effect of the magazine object or the first aid kit object.

[0163] Processor 30 continues and repeats the processes of steps S31 to S34 until the game ends. When the game ends, for example, when the player inputs a predetermined input operation to end the game (YES in step S35), the process shown in FIG. 13 ends.

[0164] (Game Progression Processing in User Terminal 100) FIG. 14 is a flowchart showing an example of the flow of the game progression process executed by the user terminal 100.

[0165] In step S41, the processor 10 receives game progress information. In step S42, the processor 10, functioning as the game progression unit 115, progresses the game based on the game progress information.

[0166] In step S43, when processor 10 accepts an item insertion operation by user 3 (YES in step S43), in step S44, processor 10 consumes virtual currency and places effect object 674 in virtual space 600B. Here, virtual currency may be purchased (charged for the game) by user 3 performing a predetermined operation on processor 10 before or during participation in the game, or may be granted to user 3 when a predetermined condition is met. The predetermined condition may be one that requires participation in the game, such as completing a quest in the game, or one that does not require participation in the game, such as answering a questionnaire. The amount of virtual currency (amount of virtual currency owned) is, for example, stored in user terminal 100 as game information 132.

[0167] In step S45, processor 10 transmits item insertion information to server 200. The item insertion information is transmitted to game play terminal 300 via server 200.

[0168] After placing the effect object 674, when a predetermined time has elapsed, the processor 10 places an item object in the virtual space 600A. In the example of Fig. 11, an obstacle object 673 is placed. That is, when the user 3 inputs a tap operation on the UI image 711C, a predetermined amount of virtual currency is consumed and the obstacle object 673 is placed.

[0169] Processor 10 continues and repeats the processing of steps S41 to S45 until the game ends. When the game ends, for example, when a player performs a predetermined input operation to end the game or when user 3 performs a predetermined input operation to leave the game midway (YES in step S46), the processing shown in FIG. 14 ends.

[0170] (Game Progression Processing in Server 200) FIG. 15 is a flowchart showing an example of the flow of the game progression process executed by the server 200.

[0171] In step S51, processor 20 receives game progress information from game play terminal 300. In step S52, processor 20, functioning as log generation unit 212, updates a log of game progress (hereinafter, play log). Note that the play log is generated by processor 20, for example, when initial placement information is received from game play terminal 300.

[0172] In step S53, the processor 20 transmits the received game progress information to each user terminal 100.

[0173] In step S54, if item insertion information has been received from any user terminal 100 (YES in step S54), in step S55, processor 20 updates the play log as log generation unit 212. In step S56, processor 20 transmits the received item insertion information to game play terminal 300.

[0174] Processor 20 continues and repeats the processes of steps S51 to S56 until the game ends. When the game ends, for example, when information indicating that the game has ended is received from game play terminal 300 (YES in step S57), in step S58, processor 20, as list generation unit 213, generates a list of users who participated in the game (user list 234) from the play log. Processor 20 stores the generated user list 234 in server 200.

[0175] FIG. 16 is a diagram showing a specific example of user list 234. The "User" column stores information (e.g., username) indicating each user who participated in the game. The "Tag" column stores information (tags) generated based on the support provided by each user to the player. In the example of FIG. 16, among the tags stored in the "Tag" column, those without quotation marks are information automatically generated by processor 20, and those with quotation marks are information manually entered by the game operator.

[0176] 16, user "AAAAA" is associated with the information of magazine, 10th floor, boss, and "defeat the boss by gifting a magazine." This indicates that, for example, in a boss battle on the 10th floor, user "AAAAAA" inserted a magazine, and avatar object 610 defeated the boss with bullets from the inserted magazine.

[0177] In addition, user "BBBBB" is associated with the information: first aid kit, 3F, weak enemy, and "recovered just before game over." This indicates that, for example, in a battle with a weak enemy on stage 3F, user "BBBBB" threw in a first aid kit, and as a result, the avatar object 610's health recovered just before it reached 0 (the game would be over).

[0178] Additionally, the user "CCCCC" is associated with the information "barricade, 5F, weak enemies, and 'barricade held back two zombies.'" This indicates that, for example, in a battle with weak enemies on the 5F stage, the user "CCCCC" threw in a barricade (obstacle object 672 in FIG. 11), and as a result, succeeded in holding back two weak enemies.

[0179] In the example of Figure 16, one support provided is associated with the username of each user 3, but the username of a user 3 who provided support multiple times is associated with tags for each of the multiple support times. It is preferable that each tag is distinguished in the user list 234. This allows a player who refers to the user list 421 using the distribution terminal 400 after the game ends to accurately understand the content of each support.

[0180] <Distribution of operation instruction data> (Distribution process in distribution terminal 400) Fig. 17 is a flowchart showing an example of the flow of distribution processing executed by distribution terminal 400. Fig. 18 is a diagram showing a specific example of a screen displayed on distribution terminal 400. Fig. 19 is a diagram showing another specific example of a screen displayed on the distribution terminal.

[0181] In step S61, processor 40, functioning as operation receiving unit 413, receives a first operation for displaying a list of users who have participated in the game (user list 234). Download screen 721 shown in FIG. 18(A) is a screen for downloading user list 234 from server 200 and displaying it on display unit 452. Download screen 721 is, as an example, a screen that is displayed immediately after an operation to start an application that executes the distribution process shown in FIG. 17 is input to distribution terminal 400.

[0182] Download screen 721 includes, for example, UI images 722 and 723. UI image 722 accepts an operation for downloading user list 234, i.e., the first operation described above. The first operation may be, for example, an operation of tapping UI image 722. UI image 723 accepts an operation for closing an application. The operation may be, for example, an operation of tapping UI image 723.

[0183] When a tap operation on the UI image 722 is accepted, in step S62, the processor 40, functioning as the communication control unit 411, acquires (receives) the user list 234 from the server 200. In step S63, the processor 40, functioning as the display control unit 412, causes the user list 234 to be displayed on the display unit 452. Specifically, the processor 40 causes the display unit 452 to display a user list screen generated based on the user list 234. As an example, the user list screen may be a user list screen 731 shown in FIG. 18(B). The user list screen 731 is made up of record images corresponding to the respective records in the user list 234. In the example of FIG. 18(B), record images 732A to 732C are shown as record images, but the number of record images is not limited to three. In the example of Figure 18(B), if the number of records in the user list 234 is greater than three (i.e., the number of users participating in the game is greater than three), the player can display other record images on the display unit 452, for example, by inputting an operation to scroll the screen (e.g., a drag operation or a flick operation) on the touch screen 45.

[0184] As an example, record images 732A to 732C include user names 733A to 733C, tag information 734A to 734C, and icons 735A to 735C, respectively. Hereinafter, when there is no need to distinguish between record images 732A to 732C, user names 733A to 733C, tag information 734A to 734C, and icons 735A to 735C, they will be referred to as "record image 732," "user name 733," "tag information 734," and "icon 735," respectively.

[0185] The user name 733 is information indicating each user who participated in the game, which is stored in the "user" column in the user list 234. The tag information 734 is information indicating a tag associated with each piece of information indicating each user who participated in the game in the user list 234. For example, the record image 732A includes "AAAAA" as the user name 733A. Therefore, the record image 732A includes, as the tag information 734A, "Magazine, 10th floor, boss, 'Defeat the boss by receiving a magazine as a gift'," which is associated with "AAAAA" in the user list 234. The icon 735 is, for example, an image set in advance by the user.

[0186] The processor 40 may store the received user list in the distribution terminal 400 (user list 421 in FIG. 7). The download screen 721 may include a UI image (not shown) for displaying the user list 421 on the display unit 452. In this example, when the UI image is tapped, the processor 40 does not download the user list 234, but reads out the user list 421, generates a user list screen from the user list 421, and displays it on the display unit 452.

[0187] In step S64, processor 40, functioning as operation receiving unit 413, receives a second operation for selecting one of the users included in user list screen 731. For example, the second operation may be an operation of tapping one of record images 732 on user list screen 731. In the example of FIG. 18(B), the player inputs a tap operation on record image 732A. That is, the player selects user "AAAAA" as the user to whom action instruction data is to be distributed.

[0188] When a tap operation on the record image 732 is received, in step S65, the processor 40, functioning as the display control unit 412, causes the motion list 422 to be displayed on the display unit 452. Specifically, the processor 40 causes the display unit 452 to display a motion list screen generated based on the motion list 422. The motion list screen may be, for example, a motion list screen 741 shown in FIG. 19. The motion list screen 741 is made up of record images corresponding to each record in the motion list 422. In the example of FIG. 19, record images 742A to 742C are shown as record images, but the number of record images is not limited to three. In the example of FIG. 19, if the number of records in the motion list 422 is more than four, the player can display other record images on the display unit 452 by, for example, inputting an operation to scroll the screen (for example, a drag operation or a flick operation) to the touch screen 45.

[0189] As an example, record images 742A to 742C include motion names 743A to 743C, motion images 744A to 744C, and UI images 745A to 745C, respectively. Hereinafter, when there is no need to distinguish between record images 742A to 742C, motion names 743A to 743C, motion images 744A to 744C, and UI images 745A to 745C, they will be referred to as "record image 7432," "motion name 743," "motion image 744," and "UI image 745," respectively.

[0190] The motion name 743 is information for identifying a motion stored in the motion list 422. The motion image 744 is an image generated from the motion data associated with each motion name in the motion list 422. As an example, the processor 40 includes an image of the avatar object 610 assuming an initial pose for each motion data as the motion image 744 in the record image 742. The motion image 744 may be a UI image that accepts a predetermined operation by the player (for example, a tap operation on the motion image 744). When the processor 40 accepts the predetermined operation, it may play a motion video in which the avatar object 610 moves based on the motion data. When the motion video ends, the processor 40 may automatically redisplay the motion list screen 741.

[0191] Note that instead of the motion image 744, the record image 742 may include a UI image including the text "motion playback," for example.

[0192] In step S66, processor 40, functioning as operation receiving unit 413, receives a third operation for selecting a motion. The third operation may be, for example, a tap operation on UI image 745. That is, UI image 745 receives an operation for selecting motion data corresponding to each record image 742. Having received the third operation, processor 40, functioning as motion identifying unit 415, identifies the motion data selected by the player.

[0193] In step S67, processor 40, functioning as display control unit 412 and voice receiving unit 414, receives voice input from the player while playing back a motion moving image in which avatar object 610 moves based on the selected motion data.

[0194] FIG. 20 is a diagram showing a specific example of voice input by player 4. As shown in FIG. 20, player 4 inputs speech 820A while playing back motion video 810A. This speech 820A is addressed to user 3 (hereinafter referred to as user 3A) whose username is "AAAAA." That is, in the example of FIG. 20, player 4 selects user 3A (first user) in step S64 and creates action instruction data addressed to user 3A. It is assumed that the user terminal 100 used by user 3A is user terminal 100A.

[0195] The spoken voice 820A is addressed to the user 3A, and is therefore based on the content of the support that the user 3A provided to the avatar object 610 (in other words, the player 4). Specifically, the user 3A inserted a magazine during a boss battle on the 10th floor stage, and the avatar object 610 defeated the boss with the bullets from the inserted magazine. Therefore, the spoken voice 820A has the content, "Thank you for giving me the magazine during the boss battle! Your timing was perfect, too! Thanks to you, AAAAA, I was able to clear it!" In this way, the spoken voice preferably includes the content of the support that the user 3A provided in the game and gratitude to the user 3.

[0196] In one aspect, player 4 creates a message to be sent to user 3 before starting voice input, i.e., before inputting the third operation into distribution terminal 400. In another aspect, the message to be sent to user 3 may be automatically generated by processor 40. Processor 40 may also superimpose a tag associated with user 3 selected by the second operation on motion video 810A.

[0197] The processor 40 converts the received voice into voice data. In step S68, the processor 40, functioning as the action instruction data generating unit 416, generates action instruction data including the voice data and the motion data of the selected motion.

[0198] In step S69, the processor 40, functioning as the communication control unit 411, distributes the generated action instruction data to the user terminal 100 (first computer) of the selected user 3 (user 3A in the example of FIG. 20). FIG. 21 is a diagram showing yet another specific example of a screen displayed on the distribution terminal 400. After executing step S68, the processor 40, functioning as the display control unit 412, causes the display unit 452 to display a distribution screen. The distribution screen may be, for example, a distribution screen 751 shown in FIG. 21(A). The distribution screen 751 includes a UI image 752 and a motion image 753A. Furthermore, as shown in FIG. 21(A), the distribution screen 751 may include information indicating users to whom the action instruction data is to be distributed.

[0199] The UI image 752 accepts an operation for delivering the action instruction data to the selected user 3. The operation may be, for example, a tap operation on the UI image 752. The motion image 753A is a UI image for accepting an operation for playing a video based on the generated action instruction data, i.e., a video based on the action instruction data generated for the user 3A. The operation may be, for example, a tap operation on the motion image 753A. Note that the UI image for accepting an operation for playing the generated video is not limited to the motion image 753A. For example, it may be a UI image including the text "Play video." The processor 40 may automatically redisplay the delivery screen 751 when the video ends.

[0200] Preferably, the distribution screen 751 further includes a UI image that accepts an operation to return to receiving voice input. The operation may be, for example, a tap operation on the UI image. By including the UI image on the distribution screen 751, the player 4 can retry voice input when the voice input fails, for example, when the player 4 makes a mistake in what he or she wants to say. Note that the UI image may also be a UI image that accepts an operation to return to selecting motion data.

[0201] When a tap operation on the UI image 752 is received, the processor 40 transmits the action instruction data together with information indicating the user 3A to the server 200. The server 200 identifies the user terminal 100 to which the action instruction data is to be transmitted, based on the information indicating the user 3A, and transmits the action instruction data to the identified user terminal 100 (i.e., the user terminal 100A).

[0202] When the transmission of the operation instruction data is completed, the processor 40 may, for example, cause the display unit 452 to display a distribution completion screen 761 shown in Fig. 21(B). For example, the distribution completion screen 761 includes UI images 762 and 763. Furthermore, as shown in Fig. 21(B), the distribution completion screen 761 may include text indicating that the transmission of the operation instruction data has been completed.

[0203] The UI image 762 accepts an operation to start creating action instruction data addressed to another user 3. The operation may be, for example, an operation of tapping the UI image 762. When the processor 40 accepts the tap operation, it causes the display unit 452 to display the user list screen again. That is, when the tap operation is accepted, the distribution process returns to step S63. At this time, the processor 40 may generate a user list screen based on the user list 421 stored in the distribution terminal 400 and display it on the display unit 452. The UI image 763 accepts an operation to end the application. The operation may be, for example, an operation of tapping the UI image 763. When the operation is accepted, the distribution process ends.

[0204] In the example described with reference to Figures 20 and 21, as shown in Figure 21(C), the distribution terminal 400 transmits action instruction data for a video addressed to user 3A (user 3 whose user name is "AAAAA") only to user terminal 100A.

[0205] FIG. 22 is a diagram showing another specific example of voice input by player 4. As shown in FIG. 22, player 4 inputs speech voice 820B while playing back motion video 810B. This speech voice 820B is addressed to user 3 (hereinafter referred to as user 3B) whose username is "BBBBB." That is, in the example of FIG. 22, player 4 inputs a tap operation on record image 732B corresponding to user 3B in step S64, thereby creating action instruction data addressed to user 3B. It is assumed that the user terminal 100 used by user 3B is user terminal 100B.

[0206] The speech voice 820B is addressed to the user 3B, and is therefore based on the content of the support that the user 3B provided to the avatar object 610 (in other words, the player 4). Specifically, in a battle with a weak enemy on the 3F stage, the user "BBBBB" threw in a first aid kit, and as a result, the avatar object 610's strength recovered just before it reached 0 (the game was over). Therefore, the speech voice 820B has the content, "Thanks to the first aid kit that BBBBB gave me, I didn't get a game over on the 3F. Thank you so much!"

[0207] 23A and 23B are diagrams showing yet another specific example of a screen displayed on distribution terminal 400. Distribution screen 751 shown in Fig. 23A includes UI image 752 and motion image 753B. When motion image 753B receives a tap operation, it plays a video based on the action instruction data generated for user 3B.

[0208] When a tap operation on the UI image 752 is received, the processor 40 transmits the action instruction data together with information indicating the user 3B to the server 200. The server 200 identifies the user terminal 100 to which the action instruction data is to be transmitted, based on the information indicating the user 3B, and transmits the action instruction data to the identified user terminal 100 (i.e., the user terminal 100B).

[0209] In the example described with reference to Figures 22 and 23, as shown in Figure 23(C), the distribution terminal 400 transmits the action instruction data for the video addressed to user 3B (user 3 whose username is "BBBBB") only to user terminal 100B.

[0210] As described above, the content of the voice based on the voice data included in the action instruction data is based on the content of the support that the user 3 provided to the player 4 in their most recent participation in the game. Since the content of the support differs for each user 3, the content of the voice will differ for each user 3. In other words, after the game ends, action instruction data including voices with different content is transmitted to the user terminals 100 of at least some of the users 3 who participated in the game.

[0211] 22 is different from the motion of the avatar object 610 in the example of FIG. 20. That is, when generating the action instruction data for user 3B, the player 4 selects motion data that is different from the motion data for user 3A. Specifically, in step S66, the player 4 inputs a tap operation on the UI image 745B to select the motion data corresponding to the record image 742B. In this way, the player 4 can vary the motion data included in the action instruction data for each user 3.

[0212] Then, the action instruction data for each user 3, including the voice data with content different for each user 3 and the motion data selected for each user 3, is transmitted only to the user terminal 100 of each user 3. In other words, the action instruction data that is unique for each user terminal 100 is transmitted to each of the user terminals 100 of the selected users 3.

[0213] FIG. 24 is a diagram showing an overview of transmission of game progress information from a game play terminal 300 to a user terminal 100. While the action instruction data for video playback in the user terminal 100 is unique for each user terminal 100, as shown in FIG. 24, the game progress information transmitted to the user terminals 100 of all users 3 participating in the game during game execution is common to each user terminal 100. In other words, the action instruction data included in the game progress information is also common to each user terminal 100. In this way, the action instruction data for video playback and the action instruction data for progressing the game can be said to be different data in terms of similarities and differences between user terminals 100 and transmission destinations.

[0214] (Video playback process in user terminal 100) FIG. 25 is a flowchart showing an example of the flow of a video playback process executed by the user terminal 100.

[0215] In step S71, the processor 10 receives the action instruction data as the video playback unit 117. In step S72, the processor 10, as the video playback unit 117, notifies the user 3 of the reception of the action instruction data. For example, the processor 10 notifies the user 3 of the reception of the action instruction data by at least one of displaying a notification image on the display unit 152, playing a notification sound from a speaker (not shown), and lighting or blinking a lighting unit (not shown) constituted by an LED (light-emitting diode) or the like.

[0216] In step S73, processor 10, functioning as operation receiving unit 111, receives a first playback operation for playing the video. The first playback operation may be, for example, an operation of tapping on a notification image. In step S74, processor 10, functioning as video playback unit 117, renders the action instruction data and plays the video. For example, processor 10 may launch an application for playing the game and play the video, or may launch a video playback application separate from the application and play the video. Hereinafter, the video will be referred to as a "thank you video."

[0217] 26 is a diagram showing a specific example of playback of a thank you video. Specifically, it is a diagram showing an example of playback of a thank you video on user terminal 100 of user 3A. In thank you video 910A played on user terminal 100, avatar object 610 utters audio 920A while performing a certain motion. In other words, processor 10 outputs audio 920A from a speaker (not shown) while playing back thank you video 910A including avatar object 610 performing a certain motion.

[0218] The motion in the thank you video 910A is based on the motion data selected by the player 4 when generating the action instruction data addressed to the user 3A, and the audio 920A is based on the audio data generated from the speech audio 820A input by the player 4 when generating the action instruction data. In other words, the audio 920A is audio including the details of the assistance provided by the user 3A in the game and gratitude for the assistance. In this way, by inputting the first playback operation, the user 3A can watch a thank you video in which the avatar object 610 speaks the details of the assistance provided by the user 3A in the game and gratitude for the assistance.

[0219] For example, after the playback of thank you video 910A has finished, user terminal 100 may display at least one UI image on touch screen 15. The UI image may be, for example, a UI image that accepts an operation to play thank you video 910A again, a UI image that accepts an operation to transition to another screen, or a UI image that accepts an operation to end an application.

[0220] Furthermore, as an example, user terminal 100 may display at least one UI image on touch screen 15 while thank you video 910A is being played. The UI image may be a plurality of UI images, each of which accepts an operation to temporarily stop or end thank you video 910A being played, or to change the scene being played.

[0221] Note that these UI images displayed during playback of thank you video 910A and after playback of thank you video 910A has stopped do not include UI images for replying to avatar object 610. In other words, thank you video 910A according to this embodiment does not include means for replying to avatar object 610.

[0222] 27 is a diagram showing another specific example of playback of a thank you video. Specifically, it is a diagram showing an example of playback of a thank you video on user terminal 100 of user 3B. In thank you video 910B played on user terminal 100, avatar object 610 utters audio 920B while performing a certain motion. In other words, processor 10 outputs audio 920B from a speaker (not shown) while playing back thank you video 910B including avatar object 610 performing a certain motion.

[0223] The motions in the thank you video 910B are based on the motion data selected by the player 4 when generating the action instruction data addressed to the user 3B, and the audio 920B is based on the audio data generated from the speech audio 820B input by the player 4 when generating the action instruction data. Therefore, in the example of FIG. 27, the motions performed by the avatar object 610 are different from those in the example of FIG. 26. Furthermore, the audio 920B is an audio including the content of the support provided by the user 3B in the game and gratitude for the support. Therefore, in the example of FIG. 27, the content of the audio 920B is different from that in the example of FIG. 26. The content of the audio 920A is different from that of the audio 920A.

[0224] In this way, the thank you video received by at least some of the user terminals 100 of the users 3 who participated in the game after the game ends is a video in which the speech content of the avatar object 610 differs for each user 3.

[0225] The processor 10 may superimpose a UI image 930 including content encouraging the user to participate in the next game on the video 910. The UI image 930 may be distributed together with the action instruction data, or may be stored in the user terminal 100 as game information 132.

[0226] In the above embodiment, an example has been described in which an avatar object 610, which moves in real time in response to the movements of an actor, is operated in a virtual space 600B that is the same as the virtual space 600A defined by the game play terminal 300, and a field of view image 660 of a field of view 640B corresponding to the position, orientation, and tilt of a virtual camera 620B placed in the virtual space 600B is displayed on the touch screen 15 of the user terminal 100. In this case, the position, orientation, etc. of the virtual camera 620B are changed in response to a swipe operation or a pinch-in / pinch-out operation on the touch screen 15 of the user terminal 100. However, the change in the position, orientation, etc. of the virtual camera 620B is not limited to this.

[0227] For example, a camera object 630 whose position, orientation, and tilt change in response to the operation of a switcher (not shown) by the operator of system 1 or a performer may be placed in virtual space 600A, and the position, orientation, etc. of a virtual camera 620B placed in virtual space 600B may be changed in response to changes in the position, orientation, etc. of camera object 630. Note that camera object 630 is displayed only on monitor 51 of HMD 500 connected to game play terminal 300 so that the performer can see it. As a result, the performer can make utterances, etc. from the camera object's line of sight.

[0228] Furthermore, the space in which the avatar object 610 and the like are operated is not limited to a virtual space, and may be a real space. The real space may be, for example, a space specified by an acquired image captured by a camera (imaging unit) 17. In this case, a space (hereinafter also referred to as an augmented reality space) in which the avatar object 610 and the like are arranged relative to the acquired image from the camera 17 may be generated, and a virtual camera 620B may be arranged in the augmented reality space, and the position, orientation, etc. of the virtual camera 620B may be changed in response to changes in the position, orientation, etc. of the camera 17.

[0229] Furthermore, the method for changing the position, orientation, etc. of virtual camera 620B may be selectable by the user. For example, a plurality of selectable viewing modes may be provided for user terminal 100. When TV mode (third viewing mode) is selected on user terminal 100, the position, orientation, etc. of virtual camera 620B may be changed in conjunction with camera object 630, whose position, orientation, etc. change in response to a switcher operation by an administrator or performer. When normal mode (first viewing mode) is selected on user terminal 100, the position, orientation, etc. of virtual camera 620B may be changed in response to a swipe operation or the like by the user. When AR mode (second viewing mode) is selected, a space (hereinafter also referred to as an augmented reality space) in which avatar object 610, etc. are arranged relative to an image acquired from camera 17 may be generated, and virtual camera 620B may be placed in the augmented reality space, and the position, orientation, etc. of virtual camera 620B may be changed in response to changes in the position, orientation, etc. of camera 17. This allows the viewpoint of the image displayed on touch screen 15 to be changed to suit the user's preferences. For example, in a predetermined area of ​​the display screen (such as the upper right area) during live streaming, a change icon labeled "AR" and a change icon labeled "TV" (an example is shown in Figure 28 described later) may be displayed, and by touching the "AR" change icon, the viewing mode may be switched from normal mode to AR mode, and by touching the "TV" change icon, the viewing mode may be switched from normal mode to TV mode, and by touching the change icon that was tapped, the viewing mode may be cancelled and switched back to normal mode (returning).

[0230] In the above embodiment, an example has been described in which an item is inserted by a user's input operation during the live distribution part (an example of the first part), causing an object corresponding to the item to be reflected (placed) in the virtual space, or changing the game situation or parameters (such as restoring stamina or replenishing ammo) in accordance with the item. However, for example, a costume object (an example of a specific object) associated with the avatar object 610 may be inserted to change the costume of the avatar object 610. In this case, on the user terminal owned by the user who inserted the costume object, the costume object of the corresponding part of the avatar object 610 is changed to the inserted costume object, and the changed costume is displayed, but this is not reflected on the user terminals owned by other users.

[0231] FIG. 28 is a diagram showing an example of a screen displayed on a user terminal during a live streaming part. The live streaming shown in FIG. 28 is assumed to be content generated by, for example, a player X as a performer (causing an avatar object 610 to move and speak). In FIG. 28, content is assumed to be live-streamed, in which the avatar object 610 moves and talks, as shown in FIG. 28(A), followed by a predetermined game (e.g., a quiz, rock-paper-scissors, etc.). In FIG. 28(A), as clothing objects of the avatar object 610, a plain T-shirt object 610a is associated with the upper body, a shorts object 610b is associated with the lower body, and a sock object 610c is associated with the feet. A user who starts watching the live streaming will view the avatar object 610 wearing the clothing shown in FIG. 28(A). Furthermore, users can change the position, orientation, etc. of a virtual camera 620B by swiping or pinching in / out on the touch screen 15 of their respective user terminals.

[0232] FIG. 28(B) shows a display example on a user terminal owned by user A when user A adds a clothing object at a certain timing during live streaming. Assume that user A selects and adds, for example, a striped T-shirt object and a white skirt object from among the multiple types of clothing objects that can be added. In this case, as shown in FIG. 28(B), the clothing objects associated with avatar object 610 are changed to a striped T-shirt object 610d and a white skirt object 610e, and the avatar object is displayed in a changed state. This can improve the user's sense of satisfaction.

[0233] Figure 28(C) shows a display example on a user terminal owned by a user other than user A (a user who has not added any costume objects) at the same timing as Figure 28(B). On the user terminals owned by other users, as shown in Figure 28(C), the costume object associated with avatar object 610 is not changed when user A adds a costume object.

[0234] When a costume object is inserted, virtual currency is consumed, just like with other objects. Also, item insertion information identifying the insertion of the costume object is sent to the server 200, updating the play log (e.g., S55 in FIG. 15 ). When the live streaming ends, a user list of users who watched the live streaming is generated (see FIG. 16 ). The user list includes information for identifying whether or not a costume object was inserted and the type of inserted costume object for each viewing user. This allows a player (e.g., player X) who references the user list using the streaming terminal 400 after the live streaming ends to accurately understand the details of the support, including the inserted costume object. The user list also includes information identifying the results of games played during the live streaming. The user list may be downloaded from the server 200 to the streaming terminal 400 by a predetermined operation as shown in FIG. 18(A) , or may be output from the server 200 and downloaded to the streaming terminal 400 after the live streaming ends without requiring a predetermined operation.

[0235] Next, modified examples of technical matters related to the thank you videos shown in FIGS. 17 to 23 will be described. Hereinafter, thank you videos will also be referred to as individual messages because they include individual messages (speech content) from the player to each user. While FIGS. 17 to 23 illustrate an example in which the display style (e.g., costume) of the avatar object displayed in the thank you video (individual message) is the same for all users, this is not limiting and the style may be different (vary) for each user. For example, the costume of the avatar object displayed in the individual message may be the same for users who did not add a costume object during the live broadcast, while for users who added a costume object during the live broadcast, the costume may be the costume object that the user added (an object different from that of users who did not add a costume object). Furthermore, while FIGS. 17 to 23 illustrate an example in which the player (performer) selects a user to whom the individual message is to be sent by operating the player (S64 in FIG. 17) when creating the individual message, this is not limiting and the users may be presented in a predetermined order without requiring any operation. 17 to 23, an example has been described in which the player (performer) selects a motion for the avatar object 610 by operation when creating an individual message (S66 in FIG. 17), but the present invention is not limited to this. Alternatively, any one of a plurality of types of motion may be determined without requiring an operation. Also, while an example has been described in FIGS. 17 to 23 in which the individual message is distributed to the destination user terminals at the timing when the player (performer) creates the individual message (S69 in FIG. 17), the present invention is not limited to this. Alternatively, the created individual message may be temporarily stored on the server side, and the stored individual message may be simultaneously distributed to the destination user terminals at a predetermined timing. Technical matters regarding individual messages modified in this respect and other respects will be described with reference to FIGS. 29 to 32.

[0236] FIG. 29 is a diagram showing an example of a recording screen display during a recording part (an example of a third part) for recording (creating) an individual message on the distribution terminal 400. In this example, when a player (e.g., player X) starts the recording part on the distribution terminal 400 (e.g., by touching the recording icon), users who watched the most recent live broadcast broadcast by the player are presented one by one in a predetermined order based on a user list for the live broadcast. When the recording of an individual message for a presented user is completed, the next user is presented in the predetermined order. This reduces the operational burden on the player, prevents time from being wasted on deciding which user to record an individual message for, and improves recording (creation) efficiency. In this embodiment, the predetermined order is determined, for example, from the top to the bottom based on a ranking of users who watched the most recent live broadcast, as described below, but is not limited to this as long as it can be an order that can be automatically determined, and may be determined based on the degree of contribution or involvement, such as the number of times an item was dropped or the number of times a comment was sent during the live broadcast (presented in descending order of degree), or based on the order of the user names, or may be determined randomly using random numbers. Furthermore, the predetermined order may be determined by someone other than the player (e.g., player X), and may be determined by processing on the server 200 side or by processing on the broadcasting terminal 400 side.

[0237] Furthermore, in this example, the motion (display action) of avatar object 610 in the individual message presented to the user is determined to be one of multiple types of motion for each user. This reduces the operational burden on the player, prevents time from being wasted deciding which motion to use, and improves creation efficiency. The motions may be specified according to a predetermined order for each of the multiple types of motions, or may be determined, for example, based on rankings (described below), the number of times items are inserted or comments are sent during live streaming, or randomly using random numbers. Furthermore, the motions may be determined by a party other than the player (e.g., player X), and may be determined by processing on the server 200 side or processing on the streaming terminal 400 side.

[0238] FIG. 29(A) shows an example of a display screen for creating an individual message for user B. It is assumed that user B is a user who did not add a costume object during the most recent live broadcast. The individual message recording screen displays user information about the user in the upper left corner, and the number of remaining users for whom individual messages should be recorded in the upper right corner. The user information includes an icon corresponding to the user, the user's name, and a ranking. The ranking is, for example, based on the user's level of contribution and involvement in the live broadcasts broadcasted by the player (e.g., player X) who is recording and creating the individual message. The higher the level of contribution and involvement, the higher the ranking. The ranking may also be, for example, a monthly ranking. In FIG. 29(A), user B's icon, user name, and first place ranking are shown. The remaining number of users is also indicated as "49 remaining." In this example, all users who watched the live broadcast are considered to be targets for recording individual messages, but this is not limiting, and the targets may be users selected by lottery from among users who watched the live broadcast, or users who have input items or sent comments. The users to be recorded may be determined by anyone other than the player (e.g., player X), and may be determined by processing on the server 200 side or by processing on the broadcasting terminal 400 side.

[0239] Additionally, below the user information, transmission history information regarding the transmission history of individual messages is displayed. The transmission history information includes the date on which a player (e.g., Player X) last sent an individual message to the user (e.g., "January 21, 2021" in FIG. 29(A)) and a transmission history icon 180. Tapping the transmission history icon 180 displays characters (text) converted by voice recognition based on the audio data of the individual message sent previously. FIG. 29(B) is an example display of the transmission history of individual messages sent to User B. FIG. 29(B) displays a character string converted based on the audio data of the individual message sent last on January 21, 2021 ("Thank you, Mr. B, for the recovery item. Mr. B...") and a character string converted based on the audio data of the individual message sent two messages ago on January 10, 2021 ("Mr. B...the attack item..."). This allows a player (e.g., Player X) to visually check the content of messages sent in the past and use it as material for consideration or reference for future recordings (e.g., to avoid saying the same thing). The process of converting messages into text using voice recognition may be performed by server 200 after a broadcast message, which will be described later, or may be performed when an individual message is recorded on distribution terminal 400. Note that by scrolling downwards using a swipe operation or the like, the text of previous individual messages can be displayed, and tapping transmission history icon 180 again will return to the screen of FIG. 29(A).

[0240] Live streaming log information is displayed below the transmission history information. Live streaming log information includes interactions the user performed during the live streaming (such as adding items or sending comments) and the tag information shown in FIG. 18(B). FIG. 29(A) displays information indicating that the user answered correctly in a quiz played as a game during the live streaming, and that the user sent a comment such as "That was fun!" to the avatar object. This allows users to check the live streaming log information and use it as a reference for future recordings. It is preferable that live streaming log information be displayed for each past live streaming. Note that accumulated live streaming log information can also be displayed by scrolling downwards using a swipe or other operation.

[0241] At the bottom of the recording screen, a recording icon 181, a retake icon 182, and a switch icon 183 are displayed. The recording icon 181 is an icon for accepting a start operation to start recording the voice spoken by a player (for example, player X) or a stop operation to stop recording. After a user in a predetermined order is presented, the first operation starts recording for that user, and the second operation ends recording for that user. When recording ends, the recording icon 181 changes to a next icon, which indicates that recording for that user is complete and presents the next user. The retake icon 182 is an icon for retaking (re-recording) an individual message for that user that has already been recorded. When the retake icon 182 is operated, the next icon that changed after recording changes back to the recording icon 181.

[0242] The switch icon 183 is an icon for switching between an information display screen displaying information such as those shown in FIGS. 29(A) and 29(B) and a motion display screen displaying the motion of the avatar object 610 that has already been determined at the time of presentation. By tapping the switch icon 183 on the information display screen, an image of the avatar object 610 moving with the determined motion can be viewed, as shown in FIG. 29(C). This allows the timing, tempo, and content of speech to be considered while taking the motion into account. Before and during recording, the information display screen and the motion display screen can be switched with each tap of the switch icon 183. Therefore, before recording, the content and timing of speech can be determined to some extent by checking the information display screen and the motion display screen, and during recording, the player (e.g., Player X) can switch between the information display screen and the motion display screen at their own timing to check and speak while recording, improving convenience. The "ON" and "OFF" on the switch icon 183 mean turning on and off the information shown in FIGS. 29(A) and 29(B). Therefore, on the information display screen, "OFF" is displayed as shown in Fig. 29(A) and on the motion display screen, "ON" is displayed as shown in Fig. 29(C). Note that user B is a user who did not add any costume objects during the most recent live broadcast. Therefore, avatar object 610 associated with plain T-shirt object 610a and shorts object 610b, which were costume objects at the start of the most recent live broadcast, will be in motion.

[0243] FIG. 29(C) shows that the recording icon 181 on the motion display screen is tapped by player X to record a voice such as "Thank you for coming to the live show, Mr. B..." Note that "..." indicates that a voice is being spoken. Although not shown, a predetermined background object is placed behind the avatar object 610. When recording is completed, the switch icon 183 changes to a play icon for confirming and playing the recorded individual message. By tapping the play icon, the recorded individual message can be played and the player can check the content, allowing them to consider whether to re-record or end the recording. By tapping the next icon, the player can complete the recording for the current user and move on to recording for the next user, and by tapping the re-record icon 182, they can re-record.

[0244] FIG. 29(D) shows an example of the creation screen when user A, who has added a costume object during a live broadcast, is presented. User information includes an icon corresponding to user A, the user's name "User A," and the fact that the user is ranked 17th. The remaining number of users is also shown as "42 remaining." The sending history information indicates that the last individual message to user A was sent on January 10, 2021. When the sending history icon 180 is tapped, the character string of the individual message sent before the last time is displayed, as shown in FIG. 29(E).

[0245] Returning to FIG. 29(D), the live streaming log information displays a costume object image 190 entered by user A, along with information indicating that the user "answered" correctly in a quiz played as a game during the live streaming, that a striped T-shirt was given as a gift, and that a white skirt was given as a gift. Furthermore, by tapping switch icon 183 on the information display screen, an image of avatar object 610 moving with a previously determined motion (different from the motion shown in FIG. 29(C)) is displayed, as shown in FIG. 29(F), allowing the player (e.g., player X) to confirm the motion. The costume of avatar object 610 when recording an individual message to a user who entered a costume object will be the costume of the costume object entered by that user. Therefore, in the case of user A, avatar object 610 associated with a costume object consisting of a striped T-shirt and a white skirt will move, as shown in FIG. 29(F).

[0246] FIG. 29(F) shows a state in which the recording icon 181 is tapped on the motion display screen and player X records a voice such as, "Thank you, Mr. A, for the costume. I really like it." Although not shown, a predetermined background object is placed behind the avatar object 610. After recording, the player can complete the recording for the current user by tapping the next icon and move on to recording for the next user. They can retake the message by tapping the retake icon 182. In this way, messages are presented to each user, and individual messages for the selected user are recorded. The recorded individual message is transmitted to the user device of the intended user. The recorded individual message shown in FIG. 29 was transmitted simultaneously at 6:00 PM on January 23, 2021.

[0247] FIG. 30 is a diagram showing an example of a display screen during a playback part (an example of the second part) for playing back individual messages on a user terminal. On a user terminal owned by user A, user A selects player X on a predetermined message receiving screen, and a list of individual messages from player X, as shown in FIG. 30(A), is displayed. In FIG. 30(A), an individual message 191 dated January 23, 2021 and an individual message 192 dated January 10, 2021 are displayed. User A touches an individual message to play the touched individual message.

[0248] FIG. 30(B) shows an example of the display and sound when an individual message 191 dated January 23, 2021 is touched and played back. In FIG. 30(B), the content recorded in FIG. 29(F) is played back. That is, an avatar object 610 associated with a clothing object consisting of a striped T-shirt and white skirt that user A uploaded during the previous live broadcast moves, and a sound such as "Thank you for the outfit, A-san. I really like it..." is output.

[0249] Next, we will explain the individual messages of user B. On the user terminal owned by user B, user B selects player X on a predetermined message receiving screen, and a list of individual messages from player X, as shown in Figure 30(C), is displayed. In Figure 30(C), an individual message 193 dated January 23, 2021, an individual message 194 dated January 21, 2021, and an individual message 195 dated January 10, 2021 are displayed.

[0250] Figure 30(D) shows an example of the display and sound when an individual message 193 dated January 23, 2021 is touched and played back. In Figure 30(D), the content recorded in Figure 29(C) is played back. That is, an avatar object 610 associated with a plain T-shirt and shorts clothing object that user A inserted during the previous live broadcast moves, and a sound such as "Mr. B, thank you for coming to the live broadcast..." is output.

[0251] 30, a predetermined background object (not shown) is placed in the background of avatar object 610, just as during recording. Also, during playback of an individual message, just as during live streaming, the position, orientation, etc. of virtual camera 620B can be changed by swiping or pinching in / out on touch screen 15 of the user terminal, and the viewing mode can be switched to the corresponding mode by touching the "AR" change icon or the "TV" change icon.

[0252] Furthermore, the individual messages include individual messages with a bonus that grants a bonus (e.g., an insertable item, object, etc.) to the user who plays the individual message when the individual message is played. For example, an individual message that is distributed after the end of a special live broadcast will be an individual message with a bonus. That is, an individual message with a bonus is distributed depending on the type of the first part. When the individual message to be recorded is an individual message with a bonus, the player will record audio taking into consideration that a bonus will be granted. Note that whether or not an individual message will have a bonus is not limited to this, and may be determined by the player at the time of recording (e.g., by setting whether or not a bonus will be granted for each user), or may be determined randomly, for example.

[0253] Individual messages with special benefits are displayed in the individual message list in a different manner from regular individual messages that do not include special benefits, for example, with a heart mark 194a attached as shown in FIG. 30(C). FIG. 31 is a diagram showing an example of a display screen during a playback part (an example of the second part) for playing individual messages with special benefits on a user terminal. User B can play and view the individual message with special benefits by touching the individual message 194 dated January 21, 2021. FIGS. 31(B) and 31(C) show examples of the display and sound when an individual message with special benefits is played.

[0254] As shown in FIG. 31(B), at a predetermined timing after playback starts, a box object with a heart mark is displayed superimposed on avatar object 610, and a voice is output saying, "Thank you for your continued support, Mr. B. As a thank you, I'm going to send you an item now. This is a present..." Next, as shown in FIG. 31(C), the box object opens and an item object (a power drink in FIG. 31(C)) is displayed, and below it an icon is displayed that allows the user to select whether or not to accept the item. If "YES" is touched, the item can be acquired. On the other hand, if "NO" is touched, a different item is presented and can be acquired.

[0255] 32 is a flowchart showing an example of the processing flow from the end of the live distribution part, through the recording part in distribution terminal 400, to the playback part in user terminal 100. In Fig. 32, the left column shows the processing in user terminal 100, the middle column shows the processing in server 200, and the right column shows the processing in distribution terminal 400.

[0256] In step S81, processing for live streaming is performed. During live streaming, the costume of avatar object 610 can be changed by input operations from the user, as shown in FIG. 28. In step S82, it is determined whether the live streaming has ended. If it is determined that the live streaming has ended, in step S83, a user list for the ended live streaming is transmitted to streaming terminal 400 owned by the player (performer) who performed the live streaming.

[0257] In the distribution terminal 400, when a player starts a recording part (for example, by touching a recording icon), an individual message recording process is performed. In step S85, users are presented in a predetermined order based on the user list from the server 200, and a motion for the presented user is determined from among multiple types of motions. Based on the result of this process, the recording screen shown in FIG. 29 is displayed, and the recording process described with reference to FIG. 29 is performed. In step S86, it is determined whether or not recording of an individual message for the presented user is complete (for example, a touch operation is performed on the next icon). If it is determined in step S86 that recording of an individual message for the presented user is complete, the recorded individual message is transmitted to the server 200. At this time, the distribution terminal 400 generates individual message data including data for identifying the user to whom the individual message is addressed, information for identifying the recorded player, and action instruction data for identifying the recorded voice (message) and motion, and transmits the generated data to the server 200. On the server 200 side, as shown in step S90, the received individual message (individual message data) is stored in a predetermined area.

[0258] In step S88 of the individual message recording process, it is determined whether the number of remaining users for which individual messages should be recorded is zero. If the number of remaining users is not zero, the process proceeds to step S85, where processing is performed to record an individual message for the next user. On the other hand, if the number of remaining users is zero, the individual message recording process is terminated.

[0259] Meanwhile, in step S91, at a predetermined time (for example, 9:00, 18:00, etc.), the server 200 broadcasts the individual messages stored at that time to the user terminals 100 of the corresponding users.

[0260] In the user terminal 100, when the playback part for displaying and playing back the message reception screen is started (for example, by touching the message viewing icon), a list of received individual messages is displayed in step S93. Based on the result of this processing, the screen shown in Fig. 30 etc. is displayed, and processing for playing back the individual messages is performed. When a displayed individual message is touched, the corresponding individual message is played back in step S94.

[0261] In step S95, it is determined whether the selected individual message is an individual message with a bonus. If it is not determined that the individual message is an individual message with a bonus, the process proceeds to step S93 when the individual message ends. On the other hand, if it is determined that the individual message is an individual message with a bonus, lottery start information is transmitted to server 200 at a predetermined timing (at the start, after a predetermined time has elapsed since the start, etc.) after the playback of the individual message begins. In response to this, server 200 performs a lottery to determine the bonus to be granted, as shown in step S96, and transmits lottery result information that can identify the determined bonus to user terminal 100, which transmitted the lottery start information. As a result, on the user terminal 100 side, the determined bonus can be notified in the individual message being played back, and granted to the user, as shown in step S97. As a result, the screen shown in FIG. 31 is displayed. When the individual message with a bonus ends, the process proceeds to step S93.

[0262] <Example of effects> In the above embodiment, the voice utterance of the avatar object 610 based on the action instruction data, i.e., the content of the utterance of the avatar object 610 in the individual message (thank you video), is different for each user, so the user feels even more delighted when viewing the individual message. This makes the user want to participate in the game again next time and view the individual message. As a result, the user's continued participation in the game can be encouraged. Furthermore, the content of the utterance of the avatar object 610 in the individual message includes the user's username, details of the user's support for the avatar object 610 in the game, and gratitude for this support. This makes the user feel that the utterance is addressed to them alone, and they feel great joy. Furthermore, supporting the avatar object 610 is an act that the user wants to receive gratitude from the avatar object 610, and the content of such an action is reflected in the content of the utterance in the individual message, which can make the user feel great joy. As a result, the user's continued participation in the game can be encouraged.

[0263] Furthermore, as shown in Fig. 28, among the multiple types of objects that may be displayed during a live broadcast, the costume objects associated with the upper and lower body parts of avatar object 610 are changeable objects, and as shown in Fig. 30(B), the costume objects of avatar object 610 that may be displayed during the playback part in which an individual message is played back are costume objects that have changed during the live broadcast. This gives users a sense of exclusivity and premium quality, which in turn increases their motivation to participate in games such as live broadcasts.

[0264] 31, the individual message includes an individual message with a benefit, and as the individual message with the benefit is played back, a benefit may be given to the user who is viewing the individual message. This may give the user not only a message but also a benefit, further increasing the motivation to participate in games such as live broadcasts.

[0265] Furthermore, the benefit to be granted is determined by lottery at a predetermined timing after the playback of the individual message with the benefit, as shown in steps S95 to S97 of Fig. 32. In this way, the benefit to be granted is determined in real time, so that the expectation as to what benefit will be granted can be maintained and increased until the end.

[0266] 28 includes motion data input by a player (performer) acting out avatar object 610, whereas data for playing individual messages shown in Fig. 30 and the like includes motion data randomly selected from multiple types of predetermined motion data. This allows for a more lively feel during live streaming, while eliminating the need for players to actually move and input motion when recording individual messages, and further eliminating the need to select motions, thereby reducing the burden on players and enabling the efficient recording (creation) of a large number of messages.

[0267] The motion data included in the individual message data is randomly determined, so the motion data will be different for each user. This can enhance the sense of exclusivity and premium for the user.

[0268] As shown in FIGS. 28 and 30 , among the multiple types of objects that can be displayed during a live broadcast, the costume object associated with avatar object 610 can be changed during the live broadcast viewed on the user terminal used by the user (user A) in response to input operations from the user. The costume object associated with avatar object 610 that can be displayed during the playback portion of an individual message to user A that can be delivered after the live broadcast is the costume object changed in response to input operations from user A during the most recent live broadcast and is different from the costume object associated with avatar object 610 that can be displayed during the playback portion of individual messages delivered to other users. This allows the user to feel a sense of satisfaction in changing the costume object during the live broadcast and to feel a similar sense of satisfaction when subsequent individual messages are played. This increases the user's motivation to change the costume object (add a costume object) during the next live broadcast, thereby livening up the live broadcast.

[0269] 28 is content based on data entered by the performer playing the avatar object 610 and live-streamed in real time, which gives the user a sense of liveness and increases the enjoyment of the game.

[0270] During the recording part on the distribution terminal 400 side, the performer playing the avatar object 610 records voices spoken to each of the multiple users, including user A who has inserted the costume object, and action instruction data including voice data that can identify the recorded voice for each of the multiple users and motion data that can identify a randomly determined motion, etc., is transmitted to the user terminals of each of the multiple users. Furthermore, as shown in Figures 29(B) and 29(E), the recorded voice is converted into a character string using voice recognition, and during the next recording part, when a voice to be spoken to the indicated user is recorded, the character string is displayed as a transmission history sent to that user. This allows the player to visually check the contents of messages previously sent to the indicated user and use them as material for consideration or reference for the content to be recorded in the future (for example, to avoid saying the same thing, to say something related to the previous time, etc.).

[0271] As shown in steps S87, S90, and S91 in Fig. 32, the recorded individual message data is temporarily saved and then simultaneously transmitted to the user terminals used by multiple users at a predetermined time. This determines the time at which the individual message can be received, thereby increasing the user's anticipation for receiving the individual message at that time and increasing interest.

[0272] <Modification> Modifications of the above-described embodiment are listed below.

[0273] (1) In the above embodiment, as shown in steps S95 to S97 of FIG. 32, an example has been described in which a benefit to be granted in association with playback of an individual message with a benefit is determined by a lottery in real time after playback begins. However, the timing of determining the benefit to be granted is not limited to this, and the benefit may be determined in advance, for example, before the individual message with a benefit is played back. For example, when saving an individual message in step S90, if the individual message includes a benefit, processing corresponding to step S96 may be performed to determine in advance the benefit to be granted when the individual message with a benefit is played back. This allows the processing load when playing back an individual message to be distributed and reduced in timing.

[0274] (2) In the above embodiment, an example has been described in which the costume object associated with avatar object 610 displayed in the first part of a live broadcast or the like is changed by being inserted in response to an input operation from the user. However, in addition to or instead of this, the costume object associated with avatar object 610 may change each time a live broadcast is performed.

[0275] For example, if a live performance by player X is broadcast every Monday starting at noon, the costume object of the avatar object 610 played by player X may be changed every week. This allows all users watching player X's live broadcast to see that the costume object of the avatar object 610 changes every week, providing excitement by wondering what costume object will be displayed this week. In this case, the costume object may be randomly determined to eliminate the need for the player to select a costume object. In this case, the costume object of the avatar object 610 in an individual message delivered after the end of a certain week's live broadcast may be the costume object planned to be associated with the avatar object 610 in the next week's live broadcast. In this case, the costume object of the avatar object 610 in the individual message for all users will be the same. This allows the individual message to notify users of the next costume object, thereby increasing their motivation to watch the next live broadcast. Even in this case, for example, if a costume object is changed by being introduced in response to an input operation from a user during a live broadcast in a certain week, the costume object of avatar object 610 in an individual message delivered to the user after the live broadcast ends may be the costume object introduced by the user (a costume object different from that of other users).

[0276] Furthermore, the costume object of avatar object 610 in an individual message delivered after the end of a certain week's live streaming may be the costume object of avatar object 610 in the ended live streaming. In this case, the costume object of avatar object 610 in individual messages to all users will be the same. Even in this case, for example, if a costume object was changed by being inserted in response to an input operation from a user during a certain week's live streaming, the costume object of avatar object 610 in an individual message delivered to that user after the end of the live streaming may be the costume object inserted by that user (which will be a costume object different from that of other users).

[0277] In addition, examples have been explained in which the costume object of the avatar object 610 displayed in an individual message is a costume object entered by the user or a costume object in a live broadcast, but this is not limited to this, and the player may be able to select from multiple types of costume objects at the start of a recording part, etc., and the costume object may be a common costume object between multiple users.

[0278] Additionally, although clothing objects associated with the upper and lower body parts of avatar object 610 have been exemplified as specific objects whose display manner may change during live streaming, etc., the specific object is not limited to this and may also be a sock object associated with the feet, a hat object associated with the head, an earring object associated with the earlobe, etc. Furthermore, the specific object may be an object different from avatar object 610, such as a background object (e.g., a picture object to hang on the wall, a stuffed animal object as a figurine, etc.).

[0279] (3) In the above embodiment, the user terminal has been described as being capable of receiving and playing individual messages with different speech content and motions between users. In addition, a common message with common speech content and motions of the avatar object 610 between users may be received and played when a user performs a playback operation. The common message may be transmitted from the server 200, for example, when a predetermined condition is met according to the progress of the game (e.g., ranking), and a message congratulating the user for meeting the predetermined condition is played. A new predetermined condition for the next stage is set for the user who has met the predetermined condition. Furthermore, in the case of a common message, a reward may be granted to the user viewing the common message (the user who has met the predetermined condition) upon playback of the common message, similar to an individual message with a reward. This reward not only conveys a message but also a reward for meeting the predetermined condition, thereby increasing motivation to meet the next predetermined condition.

[0280] (4) In the above embodiment, an example was described in which a costume object associated with avatar object 610 displayed in the first part of a live broadcast or the like is changed only on the user device of that user by inserting a costume object in response to an input operation from the user. However, this is not limited to this, and the costume object of avatar object 610 displayed on other user devices may also be changed to the inserted costume object. For example, when a special condition is met, such as when a certain user inserts a special costume object (a rare and valuable costume object) from among multiple types of costume objects, the special costume object may be associated with the avatar object 610 on the user devices of all users, causing the avatar objects to change. This can make all users watching the live broadcast feel special, thereby livening up the live broadcast.

[0281] (5) In the above embodiment, as shown in Fig. 29, when the information display screen is displayed, the motion of the avatar object 610 cannot be confirmed. However, this is not limited to this. Even when the information display screen is displayed, a motion display screen with reduced transparency may be displayed in a layer with a lower priority than the layer displaying the information display screen. This allows the player to check the motion display screen while checking the information display screen, improving convenience for the player.

[0282] (6) In the above embodiment, an example was described in which a bonus was given in response to playback of an individual message with a bonus, but this is not limited to this. All individual messages delivered after the live broadcast has ended may be individual messages with a bonus. This can increase the motivation to participate in the first part of the live broadcast, etc.

[0283] (7) In the above embodiment, the motion data included in the data of the individual message is randomly determined, and therefore the motion data is different for each user. However, the motion of the avatar object 610 in the individual message may be common to all users, and the motion data included in the data of the individual message may be common to all users.

[0284] (8) In the above embodiment, an example has been described in which individual messages with different spoken content are sent between users. However, this is not limited to the case where an individual message with different spoken content to a first user is sent to another user (a second user). It is also possible that an individual message with the same spoken content to a first user is sent to another user (a third user). Furthermore, while an example has been described in which individual messages are sent for each user, this is not limiting. For example, in a case in which users belong to one of multiple teams and play a predetermined game on a team-by-team basis, individual messages may be sent for each team, with the spoken content differing for each team. In this case, individual messages with the same spoken content are sent to users belonging to the same team.

[0285] (9) In the above embodiment, a live distribution part is exemplified as the first part, but the first part is not limited to this as long as it is content that is common to multiple users, and may be, for example, content of a live game as shown in Fig. 11 or content in which a user watches an avatar object 610 singing live. Furthermore, the first part is not limited to content that is distributed in real time, and may also be content that is pre-recorded and played back.

[0286] (10) In the above embodiment, the video of the avatar object 610 in action is displayed on the display unit 152 based on the action instruction data delivered by real-time rendering. However, the video may also be displayed based on, for example, 360-degree video data delivered by a video delivery method. In the case of 360-degree video data, an image from a viewpoint controlled according to the viewing mode may be displayed.

[0287] <Additional Notes> The matters described in the above embodiments will be supplemented below.

[0288] (Appendix 1): According to an aspect of an embodiment disclosed herein, there is provided a program for causing a first computer used by a first user to execute a game in which a plurality of users including the first user participate, the program causing the first computer to execute the following steps: in a first part (e.g., a live distribution part), displaying an image of a virtual space in which a plurality of types of objects including a moving character object are arranged, based on first data received from an external source and common to a plurality of computers used by the plurality of users; and in a second part (e.g., a playback part), displaying an image including at least the character object among the plurality of types of objects, based on second data received from an external source, the second data comprising: The data includes information for causing the character object to at least speak, and is transmitted from the outside to one or more arbitrary computers, including the first computer, among the plurality of computers used by the plurality of users, and the content of the speech is different from that of the data transmitted to at least a second computer used by a second user among the plurality of computers (e.g., Figure 29), and a specific object among the plurality of types of objects that can be displayed in the first part is an object whose display mode can be changed (e.g., Figure 28), and the display mode of the specific object that can be displayed in the second part is the display mode of the specific object that changes in the first part (e.g., Figure 30).

[0289] (Appendix 2): In (Supplementary Note 1), the specific object is an object associated with a specific part of the character object.

[0290] (Appendix 3): In (Appendix 1) or (Appendix 2), the first computer is caused to execute a step of granting a benefit to a user watching the second part in conjunction with the display of a video including the character object in the second part.

[0291] (Appendix 4): In (Supplementary Note 3), the benefit to be granted is determined at a predetermined timing after the second part starts.

[0292] (Appendix 5): In (Supplementary Note 3), the benefit to be granted is determined in advance before the second part starts.

[0293] (Appendix 6): In any of (Appendix 1) to (Appendix 5), the first data includes motion data input by an actor playing the character object, and the second data includes motion data randomly identified from a plurality of predetermined types of motion data.

[0294] (Appendix 7): In (Supplementary Note 6), the motion data included in the second data sent to the first computer is different from the motion data included in the second data sent to the second computer.

[0295] (Appendix 8): In any of (Appendix 1) to (Appendix 7), the display manner of the specific object that can be displayed in the first part can change each time the first part is performed, and the display manner of the specific object that can be displayed in the second part on the first computer is the display manner of the specific object in the first part that is performed next, and may be the same display manner as the specific object that can be displayed in the second part on the second computer.

[0296] (Appendix 9): In any of (Appendix 1) to (Appendix 8), the display mode of the specific object that can be displayed in the first part can be changed during the first part on the computer used by the user in response to an input operation from the user, and the display mode of the specific object that can be displayed in the second part on the first computer is a display mode that was changed in response to an input operation from the first user during the most recent first part, and may be a display mode that is different from the display mode of the specific object that can be displayed in the second part on the second computer.

[0297] (Appendix 10): In any of (Appendix 1) to (Appendix 9), the first data includes data input by an actor playing the character object and live-streamed in real time, and the first part includes content based on the first data live-streamed in real time.

[0298] (Appendix 11): In any of (Appendix 1) to (Appendix 10), a predetermined image based on data received from outside at a predetermined timing and common to multiple computers used by the multiple users is displayed at a timing corresponding to an input operation from the user, and the first computer is caused to execute a step of granting a benefit.

[0299] (Appendix 12): A program to be executed by an information processing device for transmitting data to a computer that executes a program described in any one of (Appendix 1) to (Appendix 11), the program executing the following steps: recording, in a third part, voice spoken by an actor playing the character object to each of a plurality of users including the first user; outputting data including voice information that can identify the voice recorded for the first user in the recording step to be sent to the first computer as the second data; and converting the voice recorded for the first user in the recording step into text, and displaying the text so that it is visible to the actor when recording the voice to be spoken to the first user in a new third part.

[0300] (Appendix 13): In (Appendix 12), the outputting step stores second data including the audio information recorded for each of the multiple users by the recording step, and outputs the second data for simultaneous transmission to computers used by each of the multiple users at a predetermined timing.

[0301] (Appendix 14): According to an aspect of an embodiment disclosed herein, there is provided a method for executing, by a first computer used by a first user, a program for executing a game in which a plurality of users including the first user participate, the method including: displaying, in a first part (e.g., a live distribution part), an image of a virtual space in which a plurality of types of objects including a moving character object are arranged, based on first data received from an external source and common to a plurality of computers used by the plurality of users; and displaying, in a second part (e.g., a playback part), an image including at least the character object among the plurality of types of objects, based on second data received from an external source, the second data being common to the character object. The data includes information for causing a character object to speak at least a voice, and is transmitted from the outside to one or more arbitrary computers, including the first computer, among a plurality of computers used by the plurality of users, and the content of the speech is different from that of the data transmitted to at least a second computer used by a second user among the plurality of computers (e.g., Figure 29), and a specific object among a plurality of types of objects that can be displayed in the first part is an object whose display mode can be changed (e.g., Figure 28), and the display mode of the specific object that can be displayed in the second part is the display mode of the specific object that changes in the first part (e.g., Figure 30).

[0302] (Appendix 15): According to an aspect of an embodiment disclosed herein, there is provided an information processing device used by a first user that executes a game in which a plurality of users including a first user participate, the information processing device including: a storage unit that stores a program for causing the information processing device to execute the game; and a control unit that controls operation of the information processing device by executing the program, wherein the control unit performs the following steps: in a first part (e.g., a live distribution part), based on first data received from an external source and common to a plurality of information processing devices used by the plurality of users, displaying an image of a virtual space in which a plurality of types of objects including a moving character object are arranged; and in a second part (e.g., a playback part), based on second data received from an external source, displaying an image of a virtual space in which a plurality of types of objects including a moving character object are arranged. and displaying a video including the character object, wherein the second data includes information for causing the character object to at least speak, and is transmitted from the outside to any one or more information processing devices among a plurality of information processing devices used by the plurality of users, including the information processing device used by the first user, and the content of the speech is different from that of the data transmitted to at least the information processing device used by the second user among the plurality of information processing devices (e.g., FIG. 29). A specific object among a plurality of types of objects that can be displayed in the first part is an object whose display mode can be changed (e.g., FIG. 28). A display mode of the specific object that can be displayed in the second part is the display mode of the specific object that changes in the first part (e.g., FIG. 30).

[0303] [Software implementation example] The control blocks (particularly control units 110, 210, 310, 410) of the user terminal 100, server 200, game play terminal 300 (HMD set 1000), and distribution terminal 400 may be realized by logic circuits (hardware) formed on an integrated circuit (IC chip) or the like, or by software.

[0304] In the latter case, the user terminal 100, the server 200, the game play terminal 300 (HMD set 1000), and the distribution terminal 400 each include a computer that executes instructions from a program, which is software that realizes each function. This computer includes, for example, one or more processors and a computer-readable recording medium that stores the program. The object of the present invention is achieved when the processor in the computer reads and executes the program from the recording medium. The processor may be, for example, a CPU (Central Processing Unit). The recording medium may be a "non-transitory tangible medium," such as a ROM (Read Only Memory), a tape, a disk, a card, a semiconductor memory, or a programmable logic circuit. The computer may also include a RAM (Random Access Memory) for expanding the program. The program may be supplied to the computer via any transmission medium capable of transmitting the program (such as a communication network or broadcast waves). Note that one aspect of the present invention may also be realized in the form of a data signal embedded in a carrier wave, in which the program is embodied by electronic transmission.

[0305] The present invention is not limited to the above-described embodiments, and various modifications are possible within the scope of the claims. Embodiments obtained by appropriately combining the technical means disclosed in different embodiments are also included in the technical scope of the present invention. [Explanation of symbols]

[0306] 1 System, 2 Network, 3, 3A, 3B User (first user), 4 Player (actor), 10, 20, 30, 40 Processor, 11, 21, 31, 41 Memory, 12, 22, 32, 42 Storage, 13, 23, 33, 43 Communication IF, 14, 24, 34, 44 Input / output IF, 15, 45 Touch screen, 17 Camera, 18 Distance measurement sensor, 51 Monitor, 52 Gaze sensor, 53 First camera, 54 Second camera, 55 Microphone, 56 Speaker, 100, 100A, 100B, 100C User terminal (computer, first computer, first information processing device), 110, 210, 310, 410 Control unit (first control unit, second control unit), 111, 311, 413 Operation reception unit, 112, 312, 412 Display control unit, 113, 313 UI control unit, 114, 314 animation generation unit, 115, 315 game progression unit, 116, 316 virtual space control unit, 117 video playback unit, 120, 220, 320, 420 memory unit (first memory unit, second memory unit), 131, 231, 331 game program (program, first program), 132, 232, 332 game information, 133, 233, 333 user information, 151, 451 input unit, 152, 452 display unit (display), 200 server, 211 communication intermediation unit, 212 log generation unit, 213 list generation unit, 234, 421 user list, 300 game play terminal (external device, second external device), 317 reaction processing unit, 400 Distribution terminal (external, first external device, computer, second information processing device), 411 communication control unit, 414 voice reception unit, 415 motion identification unit, 416 action instruction data generation unit, 422 motion list, 423 distribution program (program, second program), 540, 1020, 1021 controller, 500 HMD, 510 HMD sensor, 520 motion sensor, 530 display, 600A, 600B virtual space, 610 avatar object (character), 620A, 620B virtual camera, 631, 632, 633, 634 object, 640A, 640B field of view area, 650, 660 field of view image, 671 enemy object, 672, 673 obstacle object, 674 performance object, 691, 692 Speech, 701, 702, 703A, 70B,704A, 704B, 705, 706, 711, 711A, 711B, 711C, 711D, 722, 723, 745, 745A, 745B, 745C, 752, 762, 763, 930, 2011, 2022, 2031, 2032, 2033, 2034, 2037, 2038, 2051, 2063, 2072, 2073, 2075 UI images (message UI, UI), 721 Download screen, 731 User list screen (list), 732, 732A, 732B, 732C, 742, 742A, 742B, 742C Record images, 733, 733A, 733B, 733C User name, 734, 734A, 734B, 734C Tag information, 735, 735A, 735B, 735C Icon, 741 Motion list screen (options), 743, 743A, 743B, 743C Motion name, 744, 744A, 744B, 744C, 753 Motion image, 751 Distribution screen, 761 Distribution completion screen, 810A, 810B Motion video, 820A, 820B Speech audio, 910A, 910B Video, 920A, 920B Audio, 1000 HMD set, 1010 Object, 1030 Storage medium,

Claims

1. A computer used by a viewer user, In a first part, a step of displaying an image of a virtual space in which a plurality of types of objects including a moving character object are arranged, based on first data that is data received from an external source and is common between the computer and computers used by other users who are viewers; and in a second part after the first part, displaying a video including the character object based on second data received from an external device. the first data includes motion data input by an actor playing the character object; the second data is data transmitted to the computer from an external device, and includes a message from the character object and motion data of the character object; the motion data included in the second data is motion data randomly identified from a plurality of types of predetermined motion data, whereas the message included in the second data is a message for the user that may be different from messages for other users; a specific object among the plurality of types of objects that can be displayed in the first part is an object whose display mode can be changed in response to an input operation from a user; The display mode of the specific object that can be displayed in the second part is the display mode of the specific object after it has changed in the first part that was most recently performed.

2. A computer used by a viewer user, In a first part, a step of displaying an image of a virtual space in which a plurality of types of objects including a moving character object are arranged, based on first data that is data received from an external source and is common between the computer and computers used by other users who are viewers; and in a second part before the start of the first part, displaying a video including the character object based on second data received from an external device; the first data includes motion data input by an actor playing the character object; the second data is data transmitted to the computer from an external device, and includes a message from the character object and motion data of the character object; the motion data included in the second data is motion data randomly identified from a plurality of types of predetermined motion data, whereas the message included in the second data is a message for the user that may be different from messages for other users; a specific object among the plurality of types of objects that can be displayed in the first part is an object whose display mode can be changed each time the first part is performed; The display manner of the specific object that can be displayed in the second part will be the display manner of the specific object in the first part that will be performed next.

3. 3. The program according to claim 1, wherein the specific object is an object associated with a specific part of the character object.

4. A program as described in any one of claims 1 to 3, which causes a computer to execute a step of granting a benefit to a user watching the second part in conjunction with the display of a video including the character object in the second part.

5. A program according to any one of claims 1 to 4, wherein the motion data contained in the second data sent to a computer used by a first user among a plurality of computers used by a plurality of users may be different from the motion data contained in the second data sent to a computer used by a second user among the plurality of computers.

6. The program of claim 2, wherein the display manner of a specific object that can be displayed in the second part of a computer used by a first user among multiple computers used by multiple users may be the same as the display manner of a specific object that can be displayed in the second part of a computer used by a second user among the multiple computers.

7. The program of claim 1, wherein the display manner of a specific object that can be displayed in the second part of a computer used by a first user among multiple computers used by multiple users may be different from the display manner of a specific object that can be displayed in the second part of a computer used by a second user among the multiple computers.

8. the first data includes data input by an actor playing the character object and live-streamed in real time; 8. The program according to claim 1, wherein the first part includes content based on first data that is live-streamed in real time.

9. A program as described in any one of claims 1 to 8, which displays a specified image based on data received from outside at a specified timing and common to multiple computers used by multiple users at a timing corresponding to an input operation from the user, and causes the computer to execute a step for granting a bonus.

Images