Program, method, and viewing terminal

Abstract

To provide a program capable of diversifying modes of reaction to a user's operation, a method and a viewing terminal.SOLUTION: A program comprises: receiving display data that makes it possible to display an image in a virtual space in which objects including a character object to be operated by a performer are disposed; displaying the image in the virtual space on a touch screen based on the display data; and executing control depending on types of the objects, in response to a touch operation onto the object disposed in the virtual screen.SELECTED DRAWING: Figure 30

Description

Technical Field

[0001] The present invention relates to a program, a method, and a viewing terminal.

Background Art

[0002] According to Patent Document 1, content distributed from a content server is displayed on a viewing terminal. A viewer of the content can provide an item corresponding to a real-world gift, present, gratuity, tip, campaign, etc. to the content presenter by operating a decoration button or a donation button displayed on the display screen.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] However, in Patent Document 1, although control is performed according to the operation of the decoration button or the donation button, there is room for improvement in the case of an operation on other parts.

[0005] The present invention has been conceived in view of such circumstances, and an object thereof is to provide a program, a method, and a viewing terminal capable of diversifying the mode of reaction to a user's operation.

Means for Solving the Problems

[0006] According to an aspect of an embodiment shown in the present disclosure, a program executed in a viewing terminal including a processor, a memory, and a touch screen, the processor receiving display data for displaying an image in a virtual space in which objects including a character object operated by a performer are arranged, displaying the image in the virtual space on the touch screen based on the display data, and executing control according to the type of the object in response to a touch operation on the object arranged in the virtual space.

Effect of the Invention

[0007] According to the present invention, the modes of reaction to user operations can be diversified.

Brief Description of the Drawings

[0008]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Figure 9

Figure 10

Figure 11

Figure 12

Figure 13

Figure 14

Figure 15

Figure 16

Figure 17

Figure 18

Figure 19

Figure 20

Figure 21

Figure 22

Figure 23

Figure 24

Figure 25

Figure 26

Figure 27

Figure 28

Figure 29

Figure 30

Embodiments for Carrying Out the Invention

[0009] The system according to the present disclosure is a system for providing games to a plurality of users. Hereinafter, the system will be described with reference to the drawings. It should be noted that the present invention is not limited to these examples, but is defined by the claims, and all modifications within the meaning and scope equivalent to the claims are intended to be included in the present invention. In the following description, the same reference numerals are given to the same elements in the description of the drawings, and duplicate descriptions are not repeated.

[0010] <Overview of the operation of System 1> FIG. 1 is a diagram showing an overview of System 1 according to the present embodiment. System 1 includes a plurality of user terminals 100 (computers), a server 200, a game play terminal 300 (external device, second external device), and a distribution terminal 400 (external, first external device). In FIG. 1, as an example of a plurality of user terminals 100, user terminals 100A to 100C, in other words, three user terminals 100 are shown, but the number of user terminals 100 is not limited to the illustrated example. Also, in the present embodiment, when there is no need to distinguish between user terminals 100A to C, they are described as "user terminal 100". The user terminal 100, the game play terminal 300, and the distribution terminal 400 are connected to the server 200 via the network 2. The network 2 is composed of the Internet and various mobile communication systems constructed by wireless base stations (not shown). Examples of this mobile communication system include so-called 3G and 4G mobile communication systems, LTE (Long Term Evolution), and a wireless network (e.g., Wi-Fi (registered trademark)) that can be connected to the Internet by a predetermined access point.

[0011] (Overview of the game) In this embodiment, as an example of the game provided by system 1 (hereinafter referred to as "this game"), a game mainly played by the user of game play terminal 300 will be described. Hereinafter, the user of game play terminal 300 will be referred to as "player". The player (performer) advances the game by operating a character appearing in this game, for example. Also, in this game, the user of user terminal 100 plays a role of assisting the progress of the game by the player. Details of this game will be described later. Note that the game provided by system 1 may be any game in which a plurality of users participate and is not limited to this example.

[0012] (Game play terminal 300) The game play terminal 300 advances the game in response to input operations by the player. Also, the game play terminal 300 sequentially distributes in real time to the server 200 information (hereinafter referred to as "game progress information") generated by the player's game play.

[0013] (Server 200) The server 200 transmits the game progress information (second data) received in real time from the game play terminal 300 to the user terminal 100. Also, the server 200 mediates the transmission and reception of various information among the user terminal 100, the game play terminal 300, and the distribution terminal 400.

[0014] (Distribution terminal 400) The distribution terminal 400 generates operation instruction data (first data) in response to input operations by the user of the distribution terminal 400 and distributes the operation instruction data to the user terminal 100 via the server 200. The operation instruction data is data for playing a video on the user terminal 100, and specifically, data for operating the characters appearing in the video.

[0015] In this embodiment, as an example, the user of the distribution terminal 400 is a player of this game. Also, as an example, the video played based on the operation instruction data on the user terminal 100 is a video in which the character operated by the player in the game moves. "Movement" means moving at least a part of the character's body and includes speech. Therefore, the operation instruction data according to this embodiment includes, for example, voice data for making the character speak and motion data for moving the character's body.

[0016] As an example, the operation instruction data is transmitted to the user terminal 100 after the game ends. Details of the operation instruction data and the video played based on the operation instruction data will be described later.

[0017] (User Terminal 100) The user terminal 100 receives game progress information in real time and uses this information to generate and display a game screen. In other words, the user terminal 100 reproduces the game screen of the game being played by the player through real-time rendering. As a result, the user of the user terminal 100 can view the same game screen as the game screen that the player is viewing while playing the game at almost the same timing as the player.

[0018] Also, the user terminal 100 generates information for assisting the progress of the game by the player in response to an input operation by the user, and transmits this information to the game play terminal 300 via the server 200. Details of this information will be described later.

[0019] Also, the user terminal 100 receives operation instruction data from the distribution terminal 400, and uses this operation instruction data to generate and play a video (image). In other words, the user terminal 100 renders and plays the operation instruction data.

[0020] <Hardware Configuration of System 1> FIG. 2 is a diagram showing the hardware configuration of the user terminal 100. FIG. 3 is a diagram showing the hardware configuration of the server 200. FIG. 4 is a diagram showing the hardware configuration of the game play terminal 300. FIG. 5 is a diagram showing the hardware configuration of the distribution terminal 400.

[0021] (User Terminal 100) In this embodiment, as an example, an example in which the user terminal 100 is realized as a smartphone will be described. However, the user terminal 100 is not limited to a smartphone. For example, the user terminal 100 may be realized as a feature phone, a tablet computer, a laptop computer (so-called notebook computer), or a desktop computer. Further, the user terminal 100 may be a game device suitable for game play.

[0022] As shown in FIG. 2, the user terminal 100 includes a processor 10, a memory 11, a storage 12, a communication interface (IF) 13, an input / output IF 14, a touch screen 15 (display unit), a camera 17, and a distance measuring sensor 18. These components included in the user terminal 100 are electrically connected to each other by a communication bus. Note that the user terminal 100 may include an input / output IF 14 that can connect a display (display unit) configured separately from the user terminal 100 body instead of or in addition to the touch screen 15.

[0023] Further, as shown in FIG. 2, the user terminal 100 may be configured to be communicable with one or more controllers 1020. The controller 1020 establishes communication with the user terminal 100 according to a communication standard such as Bluetooth (registered trademark), for example. The controller 1020 may have one or more buttons or the like, and transmits an output value based on a user input operation on the buttons or the like to the user terminal 100. Further, the controller 1020 may have various sensors such as an acceleration sensor and an angular velocity sensor, and transmits output values of the various sensors to the user terminal 100.

[0024] Note that instead of, or in addition to, the user terminal 100 including the camera 17 and the distance measuring sensor 18, the controller 1020 may have the camera 17 and the distance measuring sensor 18.

[0025] For example, at the start of a game, it is desirable for the user terminal 100 to have the user using the controller 1020 input user identification information such as the user's name or login ID via the controller 1020. Thereby, the user terminal 100 can associate the controller 1020 with the user, and based on the transmission source (controller 1020) of the received output value, it can identify which user the output value belongs to.

[0026] When the user terminal 100 communicates with a plurality of controllers 1020, by each user gripping each controller 1020, it is possible to realize multiplayer on the single user terminal 100 without communicating with other devices such as the server 200 via the network 2. Also, by each user terminal 100 communicating with each other according to a wireless standard such as the wireless LAN (Local Area Network) standard (communicating without going through the server 200), it is also possible to realize local multiplayer with a plurality of user terminals 100. When realizing the above-mentioned multiplayer locally with a single user terminal 100, the user terminal 100 may further have at least a part of various functions described later provided in the server 200. Also, when realizing the above-mentioned multiplayer locally with a plurality of user terminals 100, the plurality of user terminals 100 may have the various functions described later provided in the server 200 distributed among them.

[0027] Note that even when realizing the above-mentioned multiplayer locally, the user terminal 100 may communicate with the server 200. For example, information indicating a play result such as a score or win / loss in a certain game and the user identification information may be associated and transmitted to the server 200.

[0028] Further, the controller 1020 may be configured to be detachable from the user terminal 100. In this case, a coupling portion with the controller 1020 may be provided on at least one of the surfaces of the housing of the user terminal 100. When the user terminal 100 and the controller 1020 are coupled by wire through the coupling portion, the user terminal 100 and the controller 1020 transmit and receive signals through the wire.

[0029] As shown in FIG. 2, the user terminal 100 may receive the attachment of a storage medium 1030 such as an external memory card through the input / output IF 14. Thereby, the user terminal 100 can read the programs and data recorded on the storage medium 1030. The program recorded on the storage medium 1030 is, for example, a game program.

[0030] The user terminal 100 may store in the memory 11 of the user terminal 100 a game program acquired by communicating with an external device such as the server 200, or may store in the memory 11 a game program acquired by reading from the storage medium 1030.

[0031] As described above, the user terminal 100 includes a communication IF 13, an input / output IF 14, a touch screen 15, a camera 17, and a distance measuring sensor 18 as an example of a mechanism for inputting information to the user terminal 100. Each of the above-described parts as an input mechanism can be regarded as an operation part configured to receive a user's input operation.

[0032] For example, when the operation unit is composed of at least one of the camera 17 and the distance measuring sensor 18, the operation unit detects an object 1010 near the user terminal 100, and specifies an input operation from the detection result of the object. As an example, a user's hand as the object 1010, a marker with a predetermined shape, etc. are detected, and the input operation is specified based on the color, shape, movement, or type of the object 1010 obtained as the detection result. More specifically, when the user's hand is detected from the captured image of the camera 17 by the user terminal 100, the gesture (a series of movements of the user's hand) detected based on the captured image is specified and accepted as the user's input operation. Note that the captured image may be a still image or a moving image.

[0033] Alternatively, when the operation unit is composed of the touch screen 15, the user terminal 100 specifies and accepts the user's operation performed on the input unit 151 of the touch screen 15 as the user's input operation. Alternatively, when the operation unit is composed of the communication IF 13, the user terminal 100 specifies and accepts the signal (for example, the output value) transmitted from the controller 1020 as the user's input operation. Alternatively, when the operation unit is composed of the input / output IF 14, the user terminal 100 specifies and accepts the signal output from an input device (not shown) different from the controller 1020 connected to the input / output IF 14 as the user's input operation.

[0034] (Server 200) The server 200 may be, for example, a general-purpose computer such as a workstation or a personal computer. The server 200 includes a processor 20, a memory 21, a storage 22, a communication IF 23, and an input / output IF 24. These components included in the server 200 are electrically connected to each other by a communication bus.

[0035] (Gameplay terminal 300) The game play terminal 300 may be, for example, a general-purpose computer such as a personal computer. The game play terminal 300 includes a processor 30, a memory 31, a storage 32, a communication IF 33, and an input / output IF 34. These components included in the game play terminal 300 are electrically connected to each other by a communication bus.

[0036] As shown in FIG. 4, the game play terminal 300 according to the present embodiment is included in, for example, an HMD (Head Mounted Display) set 1000. That is, it can be expressed that the HMD set 1000 is included in the system 1, and it can also be expressed that the player plays the game using the HMD set 1000. Note that the device for the player to play the game is not limited to the HMD set 1000. For example, the device may be any device capable of allowing the player to virtually experience the game. Further, the device may be realized as a smartphone, a feature phone, a tablet computer, a laptop computer (so-called notebook personal computer), or a desktop computer. Further, the device may be a game device suitable for game play.

[0037] The HMD set 1000 includes, in addition to the game play terminal 300, an HMD 500, an HMD sensor 510, a motion sensor 520, a display 530, and a controller 540. The HMD 500 includes a monitor 51, a gaze sensor 52, a first camera 53, a second camera 54, a microphone 55, and a speaker 56. The controller 540 may include the motion sensor 520.

[0038] The HMD500 can be worn on the player's head and provide a virtual space to the player during operation. More specifically, the HMD500 displays the image for the right eye and the image for the left eye on the monitor 51 respectively. When each eye of the player views each image, the player can recognize the image as a three-dimensional image based on the binocular parallax. The HMD500 can include either a so-called head-mounted display equipped with a monitor or a head-mounted device that can be worn with a terminal having a monitor such as a smartphone.

[0039] The monitor 51 is realized, for example, as a non-transmissive display device. In one aspect, the monitor 51 is disposed on the main body of the HMD500 so as to be positioned in front of both eyes of the player. Therefore, when the player views the three-dimensional image displayed on the monitor 51, the player can immerse himself in the virtual space. In one aspect, the virtual space includes, for example, an image of a background, an object that the player can operate, and a menu that the player can select. In one aspect, the monitor 51 can be realized as a liquid crystal monitor or an organic EL (Electro Luminescence) monitor provided in a so-called smartphone or other information display terminal.

[0040] In another aspect, the monitor 51 can be realized as a transmissive display device. In this case, the HMD500 can be an open type such as a glasses type instead of a sealed type that covers the player's eyes as shown in FIG. 1. The transmissive monitor 51 can be configured to be temporarily a non-transmissive display device by adjusting its transmittance. The monitor 51 may include a configuration for simultaneously displaying a part of the image constituting the virtual space and the real space. For example, the monitor 51 may display an image of the real space captured by a camera mounted on the HMD500, or may make the real space visible by setting a part of the transmittance to be high.

[0041] In a certain aspect, the monitor 51 may include a sub-monitor for displaying an image for the right eye and a sub-monitor for displaying an image for the left eye. In another aspect, the monitor 51 may be configured to integrally display the image for the right eye and the image for the left eye. In this case, the monitor 51 includes a high-speed shutter. The high-speed shutter operates to alternately display the image for the right eye and the image for the left eye so that the image is recognized only by one of the eyes.

[0042] In a certain aspect, the HMD 500 includes a plurality of light sources (not shown). Each light source is realized, for example, by an LED (Light Emitting Diode) that emits infrared rays. The HMD sensor 510 has a position tracking function for detecting the movement of the HMD 500. More specifically, the HMD sensor 510 reads a plurality of infrared rays emitted by the HMD 500 and detects the position and inclination of the HMD 500 in the real space.

[0043] In another aspect, the HMD sensor 510 may be realized by a camera. In this case, the HMD sensor 510 can detect the position and inclination of the HMD 500 by executing image analysis processing using the image information of the HMD 500 output from the camera.

[0044] In another aspect, the HMD 500 may be provided with a sensor (not shown) as a position detector instead of or in addition to the HMD sensor 510. The HMD 500 can detect its own position and inclination using the sensor. For example, when the sensor is an angular velocity sensor, a geomagnetic sensor, or an acceleration sensor, the HMD 500 can detect its own position and inclination using any of these sensors instead of the HMD sensor 510. As an example, when the sensor provided in the HMD 500 is an angular velocity sensor, the angular velocity sensor detects the angular velocity around the three axes of the HMD 500 in the real space over time. The HMD 500 calculates the temporal change of the angle around the three axes of the HMD 500 based on each angular velocity, and further calculates the inclination of the HMD 500 based on the temporal change of the angle.

[0045] The gaze sensor 52 detects the direction in which the player's right and left eyes are directed. That is, the gaze sensor 52 detects the player's line of sight. The detection of the line-of-sight direction is realized, for example, by a known eye-tracking function. The gaze sensor 52 is realized by a sensor having the eye-tracking function. In a certain aspect, it is preferable that the gaze sensor 52 includes a sensor for the right eye and a sensor for the left eye. The gaze sensor 52 may be, for example, a sensor that irradiates infrared light to the player's right and left eyes and detects the rotation angle of each eyeball by receiving the reflected light from the cornea and iris with respect to the irradiated light. The gaze sensor 52 can detect the player's line of sight based on the detected rotation angles.

[0046] The first camera 53 captures an image of the lower part of the player's face. More specifically, the first camera 53 captures an image of the player's nose, mouth, and the like. The second camera 54 captures an image of the player's eyes, eyebrows, and the like. The player-side housing of the HMD 500 is defined as the inside of the HMD 500, and the housing of the HMD 500 on the side opposite to the player is defined as the outside of the HMD 500. In a certain aspect, the first camera 53 may be disposed outside the HMD 500, and the second camera 54 may be disposed inside the HMD 500. The images generated by the first camera 53 and the second camera 54 are input to the game play terminal 300. In another aspect, the first camera 53 and the second camera 54 may be realized as a single camera, and the single camera may be used to capture an image of the player's face.

[0047] The microphone 55 converts the player's speech into an audio signal (electrical signal) and outputs it to the game play terminal 300. The speaker 56 converts the audio signal into sound and outputs it to the player. In another aspect, the HMD 500 may include earphones instead of the speaker 56.

[0048] The controller 540 is connected to the game play terminal 300 either by wire or wirelessly. The controller 540 receives input of commands from the player to the game play terminal 300. In one aspect, the controller 540 is configured to be grippable by the player. In another aspect, the controller 540 is configured to be wearable on a part of the player's body or clothing. In yet another aspect, the controller 540 may be configured to output at least any one of vibration, sound, and light based on a signal transmitted from the game play terminal 300. In yet another aspect, the controller 540 receives an operation from the player to control the position and movement of an object placed in the virtual space.

[0049] In one aspect, the controller 540 includes a plurality of light sources. Each light source is realized, for example, by an LED that emits infrared rays. The HMD sensor 510 has a position tracking function. In this case, the HMD sensor 510 reads a plurality of infrared rays emitted by the controller 540 and detects the position and orientation of the controller 540 in the real space. In another aspect, the HMD sensor 510 may be realized by a camera. In this case, the HMD sensor 510 can detect the position and orientation of the controller 540 by performing image analysis processing using the image information of the controller 540 output from the camera.

[0050] In a certain scenario, the motion sensor 520 is attached to the player's hand to detect the movement of the player's hand. For example, the motion sensor 520 detects the rotation speed, number of rotations, etc. of the hand. The detected signal is sent to the game play terminal 300. The motion sensor 520 is provided, for example, on the controller 540. In a certain scenario, the motion sensor 520 is provided, for example, on the controller 540 configured to be grippable by the player. In another scenario, for safety in the real space, the controller 540 is worn on the player's hand like a glove so that it does not easily fly away. In yet another scenario, a sensor not worn by the player may detect the movement of the player's hand. For example, the signal of a camera that photographs the player may be input to the game play terminal 300 as a signal representing the player's movement. The motion sensor 520 and the game play terminal 300 are connected to each other wirelessly as an example. In the case of wireless, the communication form is not particularly limited, and for example, Bluetooth or other known communication methods are used.

[0051] The display 530 displays an image similar to the image displayed on the monitor 51. Thereby, a user other than the player wearing the HMD 500 can also view an image similar to that of the player. The image displayed on the display 530 does not have to be a three-dimensional image, and may be an image for the right eye or an image for the left eye. Examples of the display 530 include a liquid crystal display and an organic EL monitor.

[0052] The game play terminal 300 operates a character to be an operation target of the player and advances the game based on various information acquired from each part of the HMD 500, the controller 540, and the motion sensor 520. Here, the "operation" includes moving each part of the body, changing the posture, changing the facial expression, moving, speaking, touching or moving an object arranged in the virtual space, using a weapon, a tool, etc. held by the character, and the like. That is, in this game, when the player moves each part of the body, the character also moves each part of the body in the same manner as the player. Also, in this game, the character speaks the content spoken by the player. In other words, in this game, the character is an avatar object that acts as a clone of the player. As an example, at least a part of the operation of the character may be executed by an input to the controller 540 by the player.

[0053] In the present embodiment, as an example, the motion sensor 520 is attached to both hands of the player, both feet of the player, the waist of the player, and the head of the player. As described above, the motion sensor 520 attached to both hands of the player may be provided in the controller 540. Also, the motion sensor 520 attached to the head of the player may be provided in the HMD 500. The motion sensor 520 may further be attached to both elbows and both knees of the user. By increasing the number of motion sensors 520 attached to the player, the movement of the player can be more accurately reflected in the character. Also, instead of attaching the motion sensor 520 to each part of the body, the player may wear a suit to which one or more motion sensors 520 are attached. That is, the method of motion capture is not limited to the example using the motion sensor 520.

[0054] (Distribution terminal 400) The distribution terminal 400 may be a mobile terminal such as a smartphone, a PDA (Personal Digital Assistant), or a tablet computer. Further, the distribution terminal 400 may be a so-called stationary terminal such as a desktop personal computer.

[0055] As shown in FIG. 5, the distribution terminal 400 includes a processor 40, a memory 41, a storage 42, a communication IF 43, an input / output IF 44, and a touch screen 45. Note that the distribution terminal 400 may include an input / output IF 44 that can connect a display (display unit) configured separately from the distribution terminal 400 itself instead of or in addition to the touch screen 45.

[0056] The controller 1021 may have a physical input mechanism such as one or more buttons, levers, sticks, wheels, etc. The controller 1021 transmits an output value based on an input operation input by an operator (player in this embodiment) of the distribution terminal 400 to the input mechanism to the distribution terminal 400. Further, the controller 1021 may have various sensors such as an acceleration sensor and an angular velocity sensor, and may transmit the output values of the various sensors to the distribution terminal 400. The above-described output value is received by the distribution terminal 400 via the communication IF 43.

[0057] The distribution terminal 400 may include a camera and a distance measuring sensor (both not shown). Instead of or in addition to the distribution terminal 400 including them, the controller 1021 may have a camera and a distance measuring sensor.

[0058] As described above, the distribution terminal 400 includes a communication IF 43, an input / output IF 44, and a touch screen 45 as an example of a mechanism for inputting information to the distribution terminal 400. Each of the above-described parts as an input mechanism can be regarded as an operation unit configured to receive a user's input operation.

[0059] When the operation unit is constituted by the touch screen 45, the distribution terminal 400 specifies and accepts the user operation performed on the input unit 451 of the touch screen 45 as the user input operation. Alternatively, when the operation unit is constituted by the communication IF 43, the distribution terminal 400 specifies and accepts the signal (for example, output value) transmitted from the controller 1021 as the user input operation. Alternatively, when the operation unit is constituted by the input / output IF 44, the distribution terminal 400 specifies and accepts the signal output from the input device (not shown) connected to the input / output IF 44 as the user input operation.

[0060] <Hardware Component of Each Device> The processors 10, 20, 30, 40 control the overall operations of the user terminal 100, the server 200, the game play terminal 300, and the distribution terminal 400, respectively. The processors 10, 20, 30, 40 include a CPU (Central Processing Unit), an MPU (Micro Processing Unit), and a GPU (Graphics Processing Unit). The processors 10, 20, 30, 40 read programs from the storages 12, 22, 32, 42 described later, respectively. Then, the processors 10, 20, 30, 40 expand the read programs in the memories 11, 21, 31, 41 described later, respectively. The processors 10, 20, 30 execute the expanded programs.

[0061] The memories 11, 21, 31, and 41 are main memory devices. The memories 11, 21, 31, and 41 are composed of storage devices such as ROM (Read Only Memory) and RAM (Random Access Memory). The memory 11 provides a working area for the processor 10 by temporarily storing the programs and various data read by the processor 10 from the storage 12 described later. The memory 11 also temporarily stores various data generated while the processor 10 is operating according to the program. The memory 21 provides a working area for the processor 20 by temporarily storing various programs and data read by the processor 20 from the storage 22 described later. The memory 21 also temporarily stores various data generated while the processor 20 is operating according to the program. The memory 31 provides a working area for the processor 30 by temporarily storing various programs and data read by the processor 30 from the storage 32 described later. The memory 31 also temporarily stores various data generated while the processor 30 is operating according to the program. The memory 41 provides a working area for the processor 40 by temporarily storing the programs and various data read by the processor 40 from the storage 42 described later. The memory 41 also temporarily stores various data generated while the processor 40 is operating according to the program.

[0062] In this embodiment, the programs executed by the processors 10 and 30 may be game programs of this game. In this embodiment, the program executed by the processor 40 may be a distribution program for realizing the distribution of operation instruction data. Further, the processor 10 may further execute a viewing program for realizing video playback.

[0063] In the present embodiment, the program executed by the processor 20 may be at least any one of the above-described game program, distribution program, and viewing program. The processor 20 executes at least any one of the game program, distribution program, and viewing program in response to a request or the like from at least any one of the user terminal 100, game play terminal 300, and distribution terminal 400. Note that the distribution program and the viewing program may be executed in parallel.

[0064] That is, the game program may be a program that realizes a game through the cooperation of the user terminal 100, the server 200, and the game play terminal 300. The distribution program may be a program that realizes the distribution of operation instruction data through the cooperation of the server 200 and the distribution terminal 400. The viewing program may be a program that realizes the playback of a video through the cooperation of the user terminal 100 and the server 200.

[0065] The storages 12, 22, 32, and 42 are auxiliary storage devices. The storages 12, 22, 32, and 42 are configured by storage devices such as flash memories or HDDs (Hard Disk Drives). In the storages 12 and 32, for example, various data related to games are stored. In the storage 42, various data related to the distribution of operation instruction data are stored. Also, in the storage 12, various data related to the playback of videos are stored. At least a part of the various data related to each of the game, the distribution of operation instruction data, and the playback of videos may be stored in the storage 22.

[0066] The communication IFs 13, 23, 33, and 43 control the transmission and reception of various data in the user terminal 100, server 200, game play terminal 300, and distribution terminal 400, respectively. The communication IFs 13, 23, 33, and 43 control communication using, for example, communication via a wireless LAN (Local Area Network), wired LAN, wireless LAN, or Internet communication via a mobile phone line network, and communication using short-range wireless communication or the like.

[0067] Input / output IFs 14, 24, 34, and 44 are interfaces for the user terminal 100, the server 200, the game play terminal 300, and the distribution terminal 400 to receive data input and output data, respectively. The input / output IFs 14, 24, 34, and 44 may perform data input and output via USB (Universal Serial Bus) or the like. The input / output IFs 14, 24, 34, and 44 may include physical buttons, cameras, microphones, speakers, mice, keyboards, displays, sticks, levers, and the like. Further, the input / output IFs 14, 24, 34, and 44 may include connection parts for transmitting and receiving data to and from peripheral devices.

[0068] The touch screen 15 is an electronic component that combines the input part 151 and the display part 152 (display). The touch screen 45 is an electronic component that combines the input part 451 and the display part 452. The input parts 151 and 451 are, as an example, touch-sensitive devices, and are configured by, for example, a touch pad. The display parts 152 and 452 are configured by, for example, a liquid crystal display or an organic EL (Electro-Luminescence) display.

[0069] The input parts 151 and 451 have a function of detecting the position where a user's operation (mainly physical contact operations such as touch operations, slide operations, swipe operations, and tap operations) is input with respect to the input surface, and transmitting information indicating the position as an input signal. The input parts 151 and 451 only need to be provided with a touch sensing part (not shown). The touch sensing part may adopt any method such as a capacitance method or a resistive film method.

[0070] Although not shown, the user terminal 100 and the distribution terminal 400 may each be provided with one or more sensors for specifying the holding postures of the user terminal 100 and the distribution terminal 400. This sensor may be, for example, an acceleration sensor or an angular velocity sensor.

[0071] When the user terminal 100 and the distribution terminal 400 are equipped with sensors, the processors 10 and 40 can respectively identify the holding postures of the user terminal 100 and the distribution terminal 400 from the outputs of the sensors and perform processing according to the holding postures. For example, when the user terminal 100 and the distribution terminal 400 are held vertically, the processors 10 and 40 may perform a vertical screen display that causes the vertically long images to be displayed on the display units 152 and 452. On the other hand, when the user terminal 100 and the distribution terminal 400 are held horizontally, a horizontal screen display that causes the horizontally long images to be displayed on the display unit may be performed. In this way, the processors 10 and 40 may be able to switch between the vertical screen display and the horizontal screen display according to the holding postures of the user terminal 100 and the distribution terminal 400, respectively.

[0072] <Functional Configuration of System 1> FIG. 6 is a block diagram showing the functional configurations of the user terminal 100, the server 200, and the HMD set 1000 included in the system 1. FIG. 7 is a block diagram showing the functional configuration of the distribution terminal 400 shown in FIG. 6.

[0073] The user terminal 100 has a function as an input device that receives a user's input operation and a function as an output device that outputs game images and sounds. The user terminal 100 functions as a control unit 110 and a storage unit 120 through the cooperation of a processor 10, a memory 11, a storage 12, a communication IF 13, an input / output IF 14, a touch screen 15, and the like.

[0074] The server 200 has a function of mediating the transmission and reception of various information among the user terminal 100, the HMD set 1000, and the distribution terminal 400. The server 200 functions as a control unit 210 and a storage unit 220 through the cooperation of a processor 20, a memory 21, a storage 22, a communication IF 23, and an input / output IF 24.

[0075] The HMD set 1000 (game play terminal 300) has a function as an input device that receives a player's input operation, a function as an output device that outputs game images and sounds, and a function of transmitting game progress information to the user terminal 100 in real time via the server 200. The HMD set 1000 functions as a control unit 310 and a storage unit 320 through the cooperation of a processor 30, a memory 31, a storage 32, a communication IF 33, an input / output IF 34, and the HMD 500, the HMD sensor 510, the motion sensor 520, the controller 540, etc. of the game play terminal 300.

[0076] The distribution terminal 400 has a function of generating operation instruction data and transmitting the operation instruction data to the user terminal 100 via the server 200. The distribution terminal 400 functions as a control unit 410 and a storage unit 420 through the cooperation of a processor 40, a memory 41, a storage 42, a communication IF 43, an input / output IF 44, and a touch screen 45, etc.

[0077] (Data stored in the storage unit of each device) The storage unit 120 stores a game program 131 (program), game information 132, and user information 133. The storage unit 220 stores a game program 231, game information 232, user information 233, and a user list 234. The storage unit 320 stores a game program 331, game information 332, and user information 333. The storage unit 420 stores a user list 421, a motion list 422, and a distribution program 423 (program, second program).

[0078] The game programs 131, 231, and 331 are game programs executed by the user terminal 100, the server 200, and the HMD set 1000, respectively. The game is realized by the cooperation of each device based on the game programs 131, 231, and 331. Note that the game programs 131 and 331 may be stored in the storage unit 220 and downloaded to the user terminal 100 and the HMD set 1000, respectively. In this embodiment, the user terminal 100 renders the data received from the distribution terminal 400 based on the game program 131 and plays the video. In other words, the game program 131 is also a program for playing the video using the video instruction data distributed from the distribution terminal 400. The program for playing the video may be different from the game program 131. In this case, the storage unit 120 stores a program for playing the video separately from the game program 131.

[0079] The game information 132, 232, and 332 are data referred to when the user terminal 100, the server 200, and the HMD set 1000 execute the game program, respectively. The user information 133, 233, and 333 are data related to the accounts of the users of the user terminal 100. The game information 232 is the game information 132 of each user terminal 100 and the game information 332 of the HMD set 1000. The user information 233 is the user information of the player included in the user information 133 of each user terminal 100 and the user information 333. The user information 333 is the user information 133 of each user terminal 100 and the user information of the player.

[0080] User list 234 and user list 421 are lists of users who participated in the game. User list 234 and user list 421 may include, in addition to the list of users who participated in the most recent game play by the player, the list of users who participated in each game play before the game play. Motion list 422 is a list of a plurality of pre-created motion data. The motion list 422 is, for example, a list in which motion data is associated with each piece of information (for example, motion name) for identifying each motion. The distribution program 423 is a program for realizing the distribution of operation instruction data for playing a video on the user terminal 100 to the user terminal 100.

[0081] (Functional Configuration of Server 200) The control unit 210 comprehensively controls the server 200 by executing the game program 231 stored in the storage unit 220. For example, the control unit 210 mediates the transmission and reception of various information among the user terminal 100, the HMD set 1000, and the distribution terminal 400.

[0082] The control unit 210 functions as a communication mediation unit 211, a log generation unit 212, and a list generation unit 213 according to the description of the game program 231. The control unit 210 can also function as other functional blocks (not shown) for mediating the transmission and reception of various information related to game play and the distribution of operation instruction data, and for supporting the progress of the game.

[0083] The communication mediation unit 211 mediates the transmission and reception of various information among the user terminal 100, the HMD set 1000, and the distribution terminal 400. For example, the communication mediation unit 211 transmits the game progress information received from the HMD set 1000 to the user terminal 100. The game progress information includes data indicating the movement of a character operated by the player, the parameters of the character, the items and weapons possessed by the character, information on enemy characters, and the like. The server 200 transmits the game progress information to the user terminals 100 of all users participating in the game. In other words, the server 200 transmits the common game progress information to the user terminals 100 of all users participating in the game. As a result, the game progresses in the same way as the HMD set 1000 in each of the user terminals 100 of all users participating in the game.

[0084] Also, for example, the communication mediation unit 211 transmits the information for assisting the progress of the game by the player, received from any of the user terminals 100, to the other user terminals 100 and the HMD set 1000. As an example, the information may be item information indicating an item for the player to favorably progress the game and provided to the player (character). The item information includes information (such as a user name and a user ID) indicating the user who provided the item. Further, the communication mediation unit 211 may mediate the distribution of the operation instruction data from the distribution terminal 400 to the user terminal 100.

[0085] The log generation unit 212 generates a game progress log based on the game progress information received from the HMD set 1000. The list generation unit 213 generates a user list 234 after the end of the game play. Although details will be described later, each user in the user list 234 is associated with a tag indicating the content of the support provided to the player by that user. The list generation unit 213 generates a tag based on the game progress log generated by the log generation unit 212 and associates it with the corresponding user. Note that the list generation unit 213 may also associate, as a tag, the content of the support provided to the player by each user, which is input by an operator of the game or the like using a terminal device such as a personal computer, with the corresponding user. Thereby, the content of the support provided by each user becomes more detailed. Note that when the user participates in the game, the user terminal 100 transmits information indicating the user to the server 200 based on the user's operation. For example, the user terminal 100 transmits the user ID input by the user to the server 200. That is, the server 200 holds information indicating each user for all users participating in the game. The list generation unit 213 may generate the user list 234 using this information.

[0086] (Functional Configuration of HMD Set 1000) The control unit 310 comprehensively controls the HMD set 1000 by executing the game program 331 stored in the storage unit 320. For example, the control unit 310 advances the game according to the game program 331 and the player's operation. Also, while advancing the game, the control unit 310 communicates with the server 200 as necessary to transmit and receive information. The control unit 310 may directly communicate with the user terminal 100 to transmit and receive information without going through the server 200.

[0087] The control unit 310 functions as an operation reception unit 311, a display control unit 312, a UI control unit 313, an animation generation unit 314, a game progress unit 315, a virtual space control unit 316, and a reaction processing unit 317 according to the description of the game program 331. Depending on the nature of the game being executed, the control unit 310 can also function as other functional blocks (not shown) for controlling characters appearing in the game and the like.

[0088] The operation reception unit 311 detects and receives the player's input operations. The operation reception unit 311 receives signals input from the HMD 500, the motion sensor 520, the controller 540, etc., discriminates what input operation has been made, and outputs the result to each element of the control unit 310.

[0089] The UI control unit 313 controls user interface (hereinafter referred to as UI) images to be displayed on the monitor 51, the display 530, etc. The UI image is a tool for the player to perform necessary inputs for the progress of the game on the HMD set 1000, or a tool for obtaining information output during the progress of the game from the HMD set 1000. The UI image includes, but is not limited to, for example, icons, buttons, lists, menu screens, etc.

[0090] The animation generation unit 314 generates animations showing the motions of various objects based on the control modes of the various objects. For example, the animation generation unit 314 may generate animations that express the state where an object (for example, the player's avatar object) moves as if it were actually there, moves its mouth, changes its expression, etc.

[0091] The game progress unit 315 advances the game based on the game program 331, the input operations by the player, and the actions of the avatar object according to the input operations. For example, when the avatar object performs a predetermined action, the game progress unit 315 performs a predetermined game process. Also, for example, the game progress unit 315 may receive information representing the operations of the user on the user terminal 100 and perform game processing based on the operations of the user. Further, the game progress unit 315 generates game progress information according to the progress of the game and transmits it to the server 200. The game progress information is transmitted to the user terminal 100 via the server 200. Thereby, the progress of the game in the HMD set 1000 is shared on the user terminal 100. In other words, the progress of the game in the HMD set 1000 and the progress of the game on the user terminal 100 are synchronized.

[0092] The virtual space control unit 316 performs various controls regarding the virtual space provided to the player according to the progress of the game. As an example, the virtual space control unit 316 generates various objects and arranges them in the virtual space. Also, the virtual space control unit 316 arranges a virtual camera in the virtual space. Further, the virtual space control unit 316 operates various objects arranged in the virtual space according to the progress of the game. Also, the virtual space control unit 316 controls the position and tilt of the virtual camera arranged in the virtual space according to the progress of the game.

[0093] The display control unit 312 outputs a game screen in which the processing results executed by the above-described respective elements are reflected to the monitor 51 and the display 530. The display control unit 312 may display, as the game screen, an image based on the field of view from the virtual camera arranged in the virtual space on the monitor 51 and the display 530. Also, the display control unit 312 may include the animation generated by the animation generation unit 314 in the game screen. Further, the display control unit 312 may superimpose and draw the above-described UI image controlled by the UI control unit 313 on the game screen.

[0094] The reaction processing unit 317 receives feedback on the user of the user terminal 100's reaction to the player's game play and outputs this to the player. In the present embodiment, for example, the user terminal 100 can create a comment (message) addressed to the avatar object based on the user's input operation. The reaction processing unit 317 receives the comment data of the comment and outputs this. The reaction processing unit 317 may display text data corresponding to the user's comment on the monitor 51 and the display 530, or may output voice data corresponding to the user's comment from a speaker (not shown). In the former case, the reaction processing unit 317 may superimpose and draw an image corresponding to the text data (that is, an image including the content of the comment) on the game screen.

[0095] (Functional configuration of user terminal 100) The control unit 110 comprehensively controls the user terminal 100 by executing the game program 131 stored in the storage unit 120. For example, the control unit 110 advances the game according to the game program 131 and the user's operation. Also, while advancing the game, the control unit 110 communicates with the server 200 as necessary to transmit and receive information. The control unit 110 may directly perform the transmission and reception of information with the HMD set 1000 without going through the server 200.

[0096] The control unit 110 functions as an operation reception unit 111, a display control unit 112, a UI control unit 113, an animation generation unit 114, a game progress unit 115, a virtual space control unit 116, and a video playback unit 117 according to the description of the game program 131. The control unit 110 can also function as other functional blocks (not shown) for the progress of the game according to the nature of the game being executed.

[0097] The operation reception unit 111 detects and receives the user's input operation on the input unit 151. The operation reception unit 111 determines what input operation has been made from the actions exerted by the user on the console via the touch screen 15 and other input / output IFs 14, and outputs the result to each element of the control unit 110.

[0098] For example, the operation reception unit 111 receives an input operation on the input unit 151, detects the coordinates of the input position of the input operation, and identifies the type of the input operation. The operation reception unit 111 identifies, as the type of the input operation, for example, a touch operation, a slide operation, a swipe operation, a tap operation, etc. Also, when the continuously detected input is interrupted, the operation reception unit 111 detects that the contact input has been released from the touch screen 15.

[0099] The UI control unit 113 controls the UI image to be displayed on the display unit 152 in order to construct the UI according to at least one of the user's input operation and the received game progress information. The UI image is a tool for the user to perform necessary inputs for the progress of the game on the user terminal 100, or a tool for the user to obtain information output during the progress of the game from the user terminal 100. The UI image includes, but is not limited to, for example, icons, buttons, lists, menu screens, etc.

[0100] The animation generation unit 114 generates an animation indicating the motion of various objects based on the control modes of the various objects.

[0101] The game progress unit 115 advances the game based on the game program 131, the received game progress information, and input operations by the user. When the game progress unit 115 performs a predetermined game process by an input operation by the user, the game progress unit 115 transmits information regarding the game process to the HMD set 1000 via the server 200. As a result, the predetermined game process is shared in the HMD set 1000. In other words, the progress of the game in the HMD set 1000 and the progress of the game in the user terminal 100 are synchronized. The predetermined game process is, for example, a process of providing an item to an avatar object. In this example case, the information regarding the game process is the above-described item information.

[0102] The virtual space control unit 116 performs various controls regarding the virtual space provided to the user according to the progress of the game. As an example, the virtual space control unit 116 generates various objects and arranges them in the virtual space. Also, the virtual space control unit 116 arranges a virtual camera in the virtual space. Further, the virtual space control unit 116 operates various objects arranged in the virtual space according to the progress of the game, specifically, according to the received game progress information. Also, the virtual space control unit 316 controls the position and inclination of the virtual camera arranged in the virtual space according to the progress of the game, specifically, according to the received game progress information.

[0103] The display control unit 112 outputs a game screen in which the processing results executed by the above-described respective elements are reflected, to the display unit 152. The display control unit 112 may display, as the game screen, an image based on the view from the virtual camera arranged in the virtual space provided to the user, on the display unit 152. Also, the display control unit 112 may include the animation generated by the animation generation unit 114 in the game screen. Also, the display control unit 112 may superimpose and draw the above-described UI image controlled by the UI control unit 113 on the game screen. In any case, the game screen displayed on the display unit 152 is the same game screen as the game screens displayed on other user terminals 100 and the HMD set 1000.

[0104] The video playback unit 117 analyzes (renders) the operation instruction data received from the distribution terminal 400 and plays back the video.

[0105] (Functional Configuration of Distribution Terminal 400) The control unit 410 comprehensively controls the distribution terminal 400 by executing a program (not shown) stored in the storage unit 420. For example, the control unit 410 generates operation instruction data according to the program and the operations of the user (player in this embodiment) of the distribution terminal 400, and distributes it to the user terminal 100. In addition, the control unit 410 communicates with the server 200 as necessary to send and receive information. The control unit 410 may directly communicate with the user terminal 100 without going through the server 200 to send and receive information.

[0106] The control unit 410 functions as a communication control unit 411, a display control unit 412, an operation reception unit 413, an audio reception unit 414, a motion identification unit 415, and an operation instruction data generation unit 416 according to the description of the program. The control unit 410 can also function as other functional blocks (not shown) for generating and distributing operation instruction data.

[0107] The communication control unit 411 controls the sending and receiving of information with the server 200 or the user terminal 100 via the server 200. As an example, the communication control unit 411 receives a user list 421 from the server 200. Also, as an example, the communication control unit 411 sends operation instruction data to the user terminal 100.

[0108] The display control unit 412 outputs various screens to the display unit 452 on which the processing results executed by each element are reflected. As an example, the display control unit 412 displays a screen including the received user list 234. Also, as an example, the display control unit 412 displays a screen including a motion list 422 for allowing the player to select motion data for operating an avatar object included in the operation instruction data to be distributed.

[0109] The operation reception unit 413 detects and receives the player's input operation on the input unit 151. The operation reception unit 111 determines what input operation has been made from the actions exerted by the player on the console via the touch screen 45 and other input / output IFs 44, and outputs the result to each element of the control unit 410.

[0110] For example, the operation reception unit 413 receives an input operation on the input unit 451, detects the coordinates of the input position of the input operation, and identifies the type of the input operation. As the type of the input operation, the operation reception unit 413 identifies, for example, a touch operation, a slide operation, a swipe operation, a tap operation, etc. Also, when the continuously detected input is interrupted, the operation reception unit 413 detects that the contact input has been released from the touch screen 45.

[0111] The voice reception unit 414 receives the voice generated around the distribution terminal 400 and generates the voice data of the voice. As an example, the voice reception unit 414 receives the voice uttered by the player and generates the voice data of the voice.

[0112] The motion identification unit 415 identifies the motion data selected by the player from the motion list 422 according to the player's input operation.

[0113] The operation instruction data generation unit 416 generates operation instruction data. As an example, the operation instruction data generation unit 416 generates operation instruction data including the generated voice data and the identified motion data.

[0114] Note that the functions of the HMD set 1000, server 200, and user terminal 100 shown in FIG. 6, and the functions of the distribution terminal 400 shown in FIG. 7 are merely examples. Each of the devices, namely the HMD set 1000, server 200, user terminal 100, and distribution terminal 400, may include at least a part of the functions of other devices. Further, yet another device other than the HMD set 1000, server 200, user terminal 100, and distribution terminal 400 may be a component of the system 1, and the other device may execute a part of the processing in the system 1. That is, the computer that executes the game program in the present embodiment may be any one of the HMD set 1000, server 200, user terminal 100, and distribution terminal 400, or may be realized by a combination of a plurality of these devices.

[0115] <Control Processing of Virtual Space> FIG. 8 is a flowchart showing an example of the flow of control processing of the virtual space provided to the player and the virtual space provided to the user of the user terminal 100. FIG. 9 is a diagram showing the virtual space 600A provided to the player and the field-of-view image viewed by the player according to an embodiment. FIG. 10 is a diagram showing the virtual space 600B provided to the user of the user terminal 100 and the field-of-view image viewed by the user according to an embodiment. Hereinafter, when there is no need to distinguish between the virtual spaces 600A and 600B, they will be referred to as "virtual space 600".

[0116] In step S1, the processor 30, as the virtual space control unit 316, defines the virtual space 600A shown in FIG. 9. The processor 30 defines the virtual space 600A using virtual space data (not shown). The virtual space data may be stored in the game play terminal 300, may be generated by the processor 30 based on the game program 331, or may be acquired by the processor 30 from an external device such as the server 200.

[0117] The virtual space 600 has, as an example, a full spherical structure that covers the entire 360-degree direction of the point defined as the center. In FIGS. 9 and 10, for the sake of simplicity of explanation, the upper half celestial sphere of the virtual space 600 is illustrated.

[0118] In step S2, the processor 30, as the virtual space control unit 316, places the avatar object 610 (character) in the virtual space 600A. The avatar object 610 is an avatar object associated with the player and operates according to the input operations of the player.

[0119] In step S3, the processor 30, as the virtual space control unit 316, places other objects in the virtual space 600A. In the example of FIG. 9, the processor 30 places the objects 631 to 634. Other objects may include, for example, character objects (so-called non-player characters, NPCs) that operate according to the game program 331, operation objects such as virtual hands, and objects imitating animals, plants, artifacts, natural objects, etc. that are arranged according to the progress of the game.

[0120] In step S4, the processor 30, as the virtual space control unit 316, places the virtual camera 620A in the virtual space 600A. The processor 30, as an example, places the virtual camera 620A at the position of the head of the avatar object 610.

[0121] In step S5, the processor 30 displays the view image 650 on the monitor 51 and the display 530. The processor 30 defines a view area 640A that is the view from the virtual camera 620A in the virtual space 600A according to the initial position and inclination of the virtual camera 620A. Then, the processor 30 defines a view image 650 corresponding to the view area 640A. The processor 30 outputs the view image 650 to the monitor 51 and the display 530, thereby causing the view image 650 to be displayed on the HMD 500 and the display 530.

[0122] In the example of FIG. 9, as shown in FIG. 9(A), since a part of the object 634 is included in the viewing area 640A, the view image 650 includes a part of the object 634 as shown in FIG. 9(B).

[0123] In step S6, the processor 30 transmits the initial arrangement information to the user terminal 100 via the server 200. The initial arrangement information is information indicating the initial arrangement positions of various objects in the virtual space 600A. In the example of FIG. 9, the initial arrangement information includes the initial arrangement position information of the avatar object 610 and the objects 631 to 634. The initial arrangement information can also be expressed as one of the game progress information.

[0124] In step S7, the processor 30 controls the virtual camera 620A according to the movement of the HMD500 as the virtual space control unit 316. Specifically, the processor 30 controls the orientation and tilt of the virtual camera 620A according to the movement of the HMD500, that is, the posture of the player's head. As will be described later, when the player moves the head (changes the posture of the head), the processor 30 moves the head of the avatar object 610 in accordance with this movement. The processor 30 controls the orientation and tilt of the virtual camera 620A so that, for example, the direction of the line of sight of the avatar object 610 coincides with the direction of the line of sight of the virtual camera 620A. In step S8, the processor 30 updates the view image 650 in response to the change in the orientation and tilt of the virtual camera 620A.

[0125] In step S9, the processor 30 moves the avatar object 610 according to the movement of the player as the virtual space control unit 316. As an example, the processor 30 moves the avatar object 610 in the virtual space 600A in response to the player's movement in the real space. In addition, the processor 30 moves the head of the avatar object 610 in the virtual space 600A in response to the player's movement of the head in the real space.

[0126] In step S10, as the virtual space control unit 316, the processor 30 moves the virtual camera 620A so as to follow the avatar object 610. That is, even when the avatar object 610 moves, the virtual camera 620A is always at the position of the head of the avatar object 610.

[0127] The processor 30 updates the view image 650 according to the movement of the virtual camera 620A. That is, the processor 30 updates the viewing area 640A according to the posture of the player's head and the position of the virtual camera 620A in the virtual space 600A. As a result, the view image 650 is updated.

[0128] In step S11, the processor 30 transmits the operation instruction data of the avatar object 610 to the user terminal 100 via the server 200. The operation instruction data here includes at least any one of the motion data capturing the player's operation during the virtual experience (for example, during the game play), the voice data of the voice spoken by the player, and the operation data indicating the content of the input operation on the controller 540. When the player is playing the game, the operation instruction data is transmitted to the user terminal 100 as, for example, game progress information.

[0129] The processes of steps S7 to S11 are continuously and repeatedly executed while the player is playing the game.

[0130] In step S21, the processor 10 of the user terminal 100 of user 3 defines the virtual space 600B shown in FIG. 10 as the virtual space control unit 116. The processor 10 defines the virtual space 600B using virtual space data (not shown). The virtual space data may be stored in the user terminal 100, may be generated by the processor 10 based on the game program 131, or may be acquired by the processor 10 from an external device such as the server 200.

[0131] In step S22, the processor 10 receives initial arrangement information. In step S23, the processor 10, as the virtual space control unit 116, arranges various objects in the virtual space 600B according to the initial arrangement information. In the case of the example in FIG. 10, the various objects are the avatar object 610 and the objects 631 to 634.

[0132] In step S24, the processor 10, as the virtual space control unit 116, arranges the virtual camera 620B in the virtual space 600B. As an example, the processor 10 arranges the virtual camera 620B at the position shown in FIG. 10(A).

[0133] In step S25, the processor 10 displays the view image 660 on the display unit 152. The processor 10 defines a viewing area 640B, which is the view from the virtual camera 620B in the virtual space 600B, according to the initial position and tilt of the virtual camera 620B. Then, the processor 10 defines a view image 660 corresponding to the viewing area 640B. The processor 10 causes the view image 660 to be displayed on the display unit 152 by outputting the view image 660 to the display unit 152.

[0134] In the example of FIG. 10, as shown in FIG. 10(A), since the avatar object 610 and the object 631 are included in the viewing area 640B, the view image 660 includes the avatar object 610 and the object 631 as shown in FIG. 10(B).

[0135] In step S26, the processor 10 receives operation instruction data. In step S27, the processor 10, as the virtual space control unit 116, moves the avatar object 610 in the virtual space 600B according to the operation instruction data. In other words, the processor 10 reproduces a video in which the avatar object 610 is operating by real-time rendering.

[0136] In step S28, the processor 10 controls the virtual camera 620B as the virtual space control unit 116 according to the operation of the user received as the operation reception unit 111. In step S29, the processor 10 updates the view image 660 in response to the change in the position, orientation, and tilt of the virtual camera 620B in the virtual space 600B. Note that in step S28, the processor 10 may automatically control the virtual camera 620B according to the movement of the avatar object 610, for example, the movement of the avatar object 610 or the change in its orientation. For example, the processor 10 may automatically move the virtual camera 620B or change its orientation and tilt so as to always photograph the avatar object 610 from the front. Also, as an example, the processor 10 may automatically move the virtual camera 620B or change its orientation and tilt so as to always photograph the avatar object 610 from the rear according to the movement of the avatar object 610.

[0137] In this way, in the virtual space 600A, the avatar object 610 operates according to the movement of the player. The movement instruction data indicating this movement is transmitted to the user terminal 100. In the virtual space 600B, the avatar object 610 operates according to the received movement instruction data. Thereby, in the virtual space 600A and the virtual space 600B, the avatar object 610 performs the same operation. In other words, the user 3 can visually recognize the operation of the avatar object 610 according to the movement of the player using the user terminal 100.

[0138] <Game Outline> FIG. 11 is a diagram showing another example of the view image displayed on the user terminal 100. Specifically, it is a diagram showing an example of the game screen of the game (this game) executed by the system 1 that the player is playing.

[0139] As an example, this game features an avatar object 610 that operates weapons such as guns and knives, and a plurality of enemy objects 671 that are NPCs. These are introduced into a virtual space 600, and the avatar object 610 is made to engage in battles with the enemy objects 671. Various game parameters such as the physical strength of the avatar object 610, the number of available magazines, the remaining ammunition of the gun, and the remaining number of enemy objects 671 are updated according to the progress of the game.

[0140] This game has a plurality of stages prepared. By fulfilling the predetermined achievement conditions associated with each stage, the player can clear the stage. Examples of the predetermined achievement conditions may include, for instance, defeating all the enemy objects 671 that appear, defeating the boss object among the enemy objects 671 that appear, obtaining a predetermined item, reaching a predetermined location, and other conditions that can be established. These achievement conditions are defined within the game program 131. In this game, in accordance with the content of the game, when the achievement conditions are met, the player clears the stage. In other words, the victory of the avatar object 610 over the enemy object 671 (the win or loss between the avatar object 610 and the enemy object 671) is determined. In contrast, for example, when the game executed by the system 1 is a racing game or the like, when the condition of reaching the goal is met, the ranking of the avatar object 610 is determined.

[0141] In this game, in order to share the virtual space between the HMD set 1000 and the plurality of user terminals 100, game progress information is live-streamed to the plurality of user terminals 100 at predetermined time intervals. As a result, on the touch screen 15 of the user terminal 100 while the game is being viewed, a view image of the viewing area defined by the virtual camera 620B corresponding to the user terminal 100 is displayed. Also, in the upper right and upper left sections of the view image, parameter images representing various parameters such as the physical strength of the avatar object 610, the number of available magazines, the remaining ammunition of the gun, and the remaining number of enemy objects 671 are superimposed and displayed. This view image can also be referred to as the game screen.

[0142] As described above, the game progress information includes motion data capturing the player's actions, voice data of the voice spoken by the player, and operation data indicating the content of the input operation on the controller 540. These data are, namely, information for specifying the position, posture, orientation, etc. of the avatar object 610, information for specifying the position, posture, orientation, etc. of the enemy object 671, and information for specifying the position, etc. of other objects (for example, obstacle objects 672, 673). The processor 10 specifies the position, posture, orientation, etc. of each object by analyzing (rendering) the game progress information.

[0143] The game information 132 includes data of various objects such as the avatar object 610, the enemy object 671, and the obstacle objects 672, 673. The processor 10 updates the position, posture, orientation, etc. of each object by using the data and the analysis result of the game progress information. Thereby, the game progresses, and each object in the virtual space 600B moves in the same manner as each object in the virtual space 600A. Specifically, in the virtual space 600B, each object including the avatar object 610 operates based on the game progress information regardless of whether there is an operation from the user to the user terminal 100.

[0144] On the touch screen 15 of the user terminal 100, as an example, the UI images 701 and 702 are displayed superimposed on the view image. The UI image 701 is a UI image that receives an operation for displaying, on the touch screen 15, a UI image 711 that receives an item input operation for assisting the avatar object 610 from the user 3. The UI image 702 is a UI image that receives an operation for displaying, on the touch screen 15, a UI image (described later) that receives an operation for inputting and transmitting a comment to the avatar object 610 (in other words, the player 4) from the user 3. The operations received by the UI images 701 and 702 may be, for example, operations of tapping the UI images 701 and 702.

[0145] When the UI image 701 is tapped, the UI image 711 is displayed superimposed on the view image. The UI image 711 includes, for example, a UI image 711A with a magazine icon drawn, a UI image 711B with a first aid kit icon drawn, a UI image 711C with a warning triangle icon drawn, and a UI image 711D with a barricade icon drawn. The item input operation corresponds to, for example, an operation of tapping any of the UI images.

[0146] As an example, when the UI image 711A is tapped, the remaining ammunition of the gun used by the avatar object 610 increases. When the UI image 711B is tapped, the physical strength of the avatar object 610 is restored. When the UI images 711C and 711D are tapped, obstacle objects 672, 673 that obstruct the movement of the enemy object 671 are arranged in the virtual space. The obstacle objects 672, 673 may be such that one obstructs the movement of the enemy object 671 more than the other.

[0147] The processor 10 transmits item input information indicating that an item input operation has been performed to the server 200. The item input information includes at least information for identifying the type of item specified by the item input operation. The item input information may include other information about the item, such as information indicating the position where the item is to be placed. The item input information is transmitted to other user terminals 100 and the HMD set 1000 via the server 200.

[0148] FIG. 12 is a diagram showing another example of the view image displayed on the user terminal 100. Specifically, it is a diagram showing an example of the game screen of this game, and is a diagram for explaining the communication between the player during the game play and the user terminal 100.

[0149] In the example of FIG. 12(A), the user terminal 100 causes the avatar object 610 to execute a speech 691. Specifically, the user terminal 100 causes the avatar object 610 to execute a speech 691 according to the voice data included in the game progress information. The content of the speech 691 is what the player 4 said, "I'm out of bullets!". That is, the content of the speech 691 conveys to each user that due to the magazine being 0 and the number of bullets loaded in the gun being 1, it seems that the means to attack the enemy object 671 will be lost.

[0150] In FIG. 12(A), a speech bubble is used to visually show the speech of the avatar object 610, but actually, voice is output from the speaker of the user terminal 100. In addition to the voice output, a speech bubble (i.e., a speech bubble including the text of the voice content) shown in FIG. 12(A) may be displayed in the field-of-view image. This is the same for the speech 692 described later.

[0151] When receiving a tap operation on the UI image 702, the user terminal 100, as shown in FIG. 12(B), superimposes and displays the UI images 705 and 706 (message UI) on the field-of-view image. The UI image 705 is a UI image that displays comments for the avatar object 610 (in other words, the player). The UI image 706 is a UI image that receives a comment sending operation from the user 3 to send the input comment.

[0152] As an example, when the user terminal 100 receives a tap operation on the UI image 705, it displays a UI image imitating a keyboard (not shown, hereinafter simply referred to as "keyboard") on the touch screen 15. The user terminal 100 displays the text corresponding to the user's input operation on the keyboard on the UI image 705. In the example of FIG. 12(B), the text "Send the magazine" is displayed on the UI image 705.

[0153] After the user inputs text, for example, when the user terminal 100 receives a tap operation on the UI image 706, the user terminal 100 sends comment information including information indicating the input content (text content) and information indicating the user to the server 200. The comment information is sent to other user terminals 100 and the HMD set 1000 via the server 200.

[0154] The UI image 703A is a UI image indicating the user name of the user who sent the comment, and the UI image 704A is a UI image indicating the content of the comment sent by the user. In the example of FIG. 12(B), a user with the user name "BBBBB" uses his or her own user terminal 100 to send comment information with the content "Dangerous!", and thus the UI images 703A and 704A are displayed. The UI images 703A and 704A are displayed on the touch screen 15 of all user terminals 100 participating in this game and on the monitor 51 of the HMD 500. Note that the UI images 703A and 704A may be one UI image. That is, a user name and the content of a comment may be included in one UI image.

[0155] In the example of FIG. 12(C), the user with the user name "AAAAA", who is the user of the user terminal 100 shown in FIG. 12, inputs and sends a comment as described above, and thus the UI images 703B and 704B are displayed on the touch screen 15. The UI image 703B includes the user name "AAAAA", and the UI image 704B includes the comment "Send the magazine!" input in the example of FIG. 12(B).

[0156] In the example of FIG. 12(C), the user "AAAAA" further inputs a tap operation on the UI image 701, causes the UI image 711 to be displayed on the touch screen 15, and then inputs a tap operation on the UI image 711A. The view image 611 after that is shown. That is, as a result of item input information indicating a magazine being transmitted from the user terminal 100 of the user "AAAAA" to other user terminals 100 and the HMD set 1000, the user terminals 100 and the HMD set 1000 have placed the production object 674 (described later) in the virtual space 600. As an example, after the elapsed time indicated by the item input information has elapsed, the user terminals 100 and the HMD set 1000 execute a production related to the production object 674 and execute a process of activating the effect of the item object.

[0157] In the example of FIG. 12(D), due to the execution of the process of activating the effect of the item object, the number of magazines has increased from 0 to 1. As a result, the player speaks "Thank you!" to the user "AAAAA", and the voice data of this speech is transmitted to each user terminal 100. Thereby, each user terminal 100 outputs the voice "Thank you!" as the speech 692 of the avatar object 610.

[0158] As described above, in this game, communication between the user and the avatar object 610 is realized by the output of the speech voice of the avatar object 610 based on the player's speech and the input of comments by each user.

[0159] (Game progress processing in the game play terminal 300) FIG. 13 is a flowchart showing an example of the flow of game progress processing executed by the game play terminal 300.

[0160] In step S31, the processor 30, as the game progress unit 315, advances the game based on the game program 331 and the player's movement. In step S32, the processor 30 generates game progress information and distributes it to the user terminal 100. Specifically, the processor 30 transmits the generated game progress information to each user terminal 100 via the server 200.

[0161] In step S33, when the processor 30 receives item input information (YES in S33), in step S34, based on the item input information, it places an item object in the virtual space 600A. As an example, before placing the item object, the processor 30 places the effect object 674 in the virtual space 600A (see Fig. 11(C)). The effect object 674 may be, for example, an object imitating a present box. As an example, after the elapsed time indicated by the item input information has elapsed, the processor 30 may execute an effect related to the effect object 674. The effect may be, for example, an animation of the lid of the present box opening. After executing the animation, the processor 30 executes a process of activating the effect of the item object. For example, in the example of Fig. 11(D), an obstacle object 673 is placed.

[0162] After executing the animation, the processor 30 may place an item object corresponding to the tapped UI image in the virtual space 600A. For example, when a tap operation is performed on the UI image 711A, after executing the animation, the processor 30 places a magazine object indicating a magazine in the virtual space 600A. Also, when a tap operation is performed on the UI image 711B, after executing the animation, the processor 30 places a first aid kit object indicating a first aid kit in the virtual space 600A. The processor 30 may execute a process of activating the effect of the magazine object or the first aid kit object when the avatar object 610 moves to the position of the magazine object or the first aid kit object, for example.

[0163] The processor 30 continues and repeats the processes of steps S31 to S34 until the game ends. When the game ends, for example, when the player inputs a predetermined input operation to end the game (YES in step S35), the process shown in FIG. 13 ends.

[0164] (Game progress processing in the user terminal 100) FIG. 14 is a flowchart showing an example of the flow of game progress processing executed in the user terminal 100.

[0165] In step S41, the processor 10 receives game progress information. In step S42, the processor 10 advances the game based on the game progress information as the game progress unit 115.

[0166] In step S43, when the processor 10 receives an item input operation by the user 3 (YES in step S43), in step S44, the processor 10 consumes virtual currency and arranges the effect object 674 in the virtual space 600B. Here, the virtual currency may be purchased (charged for this game) by the user 3 performing a predetermined operation on the processor 10 before participating in the game or during the game, or may be given to the user 3 when a predetermined condition is satisfied. The predetermined condition may be something necessary for participating in this game, such as clearing a quest in this game, or may be something unnecessary for participating in this game, such as answering a questionnaire. The amount of virtual currency (the amount of virtual currency owned) is stored in the user terminal 100 as game information 132 as an example.

[0167] In step S45, the processor 10 transmits the item input information to the server 200. The item input information is transmitted to the game play terminal 300 via the server 200.

[0168] After the placement of the production object 674, when a predetermined time has elapsed, the processor 10 places the item object in the virtual space 600A. In the example of FIG. 11, the obstacle object 673 is placed. That is, when the user 3 inputs a tap operation on the UI image 711C, a predetermined amount of virtual currency is consumed and the obstacle object 673 is placed.

[0169] The processor 10 continues and repeats the processing of steps S41 to S45 until the game ends. When the game ends, for example, when the player performs a predetermined input operation to end the game, or when the user 3 performs a predetermined input operation to leave the game midway (YES in step S46), the processing shown in FIG. 14 ends.

[0170] (Game progress processing in the server 200) FIG. 15 is a flowchart showing an example of the flow of game progress processing executed in the server 200.

[0171] In step S51, the processor 20 receives game progress information from the game play terminal 300. In step S52, the processor 20 updates the game progress log (hereinafter referred to as the play log) as the log generation unit 212. Note that, as an example, the play log is generated by the processor 20 when receiving the initial placement information from the game play terminal 300.

[0172] In step S53, the processor 20 transmits the received game progress information to each user terminal 100.

[0173] In step S54, when item input information is received from any of the user terminals 100 (YES in step S54), in step S55, the processor 20 updates the play log as the log generation unit 212. In step S56, the processor 20 transmits the received item input information to the game play terminal 300.

[0174] The processor 20 continues and repeats the processes of steps S51 to S56 until the game ends. When the game ends, for example, when receiving information indicating that the game has ended from the game play terminal 300 (YES in step S57), in step S58, the processor 20, as the list generation unit 213, generates a list of users who participated in the game (user list 234) from the play log. The processor 20 stores the generated user list 234 in the server 200.

[0175] FIG. 16 is a diagram showing a specific example of the user list 234. In the "User" column, information indicating each user who participated in the game (for example, user name) is stored. In the "Tag" column, information (tag) generated based on the support provided by each user to the player is stored. In the example of FIG. 16, among the tags stored in the "Tag" column, those without square brackets are information automatically generated by the processor 20, and those with square brackets are information manually input by the game operator.

[0176] In the example of FIG. 16, the user "AAAAA" is associated with the information "Magazine, 10F, Boss, 'Victory over the Boss with the Magazine Present'". This indicates that, for example, in the boss battle on the stage of 10F, the user "AAAAA" threw in a magazine, and the avatar object 610 won against the boss with the bullets of the thrown-in magazine.

[0177] Also, the user "BBBBB" is associated with the information "First Aid Kit, 3F, Mob, 'Recovery Just Before Game Over'". This indicates that, for example, in the battle against the mob enemies on the stage of 3F, the user "BBBBB" threw in a first aid kit, and as a result, the physical strength of the avatar object 610 recovered just before reaching 0 (game over).

[0178] In addition, for user "CCCCC", information such as "barricade", "5F", "mob", and "Use the barricade to stop two zombies" is associated. This indicates that, for example, in the battle against mob enemies on the stage of "5F", user "CCCCC" deployed a barricade (obstacle object 672 in FIG. 11), and as a result, successfully stopped the feet of two mob enemies.

[0179] In the example of FIG. 16, for each user name of user 3, one piece of support provided is associated. However, for the user names of user 3 who provided support multiple times, tags for each instance of the multiple supports are associated. In user list 234, it is preferable that these respective tags are distinguishable. This enables players who refer to user list 421 using distribution terminal 400 after the game ends to accurately understand the details of each support.

[0180] <Distribution of Operation Instruction Data> (Distribution Processing in Distribution Terminal 400) FIG. 17 is a flowchart showing an example of the flow of distribution processing executed by distribution terminal 400. FIG. 18 is a diagram showing a specific example of the screen displayed on distribution terminal 400. FIG. 19 is a diagram showing another specific example of the screen displayed on the distribution terminal.

[0181] In step S61, processor 40, acting as operation reception unit 413, receives a first operation for displaying a list of users (user list 234) who participated in the game. The download screen 721 shown in FIG. 18(A) is a screen for downloading user list 234 from server 200 and displaying it on display unit 452. As an example, the download screen 721 is the screen displayed immediately after an activation operation of the application that executes the distribution processing shown in FIG. 17 is input to distribution terminal 400.

[0182] The download screen 721 includes, as an example, UI images 722 and 723. The UI image 722 accepts an operation for downloading the user list 234, that is, the first operation described above. The first operation may be, for example, an operation of tapping the UI image 722. The UI image 723 accepts an operation for ending the application. The operation may be, for example, an operation of tapping the UI image 723.

[0183] When receiving a tap operation on the UI image 722, in step S62, the processor 40, as the communication control unit 411, acquires (receives) the user list 234 from the server 200. In step S63, the processor 40, as the display control unit 412, causes the display unit 452 to display the user list 234. Specifically, the processor 40 causes the display unit 452 to display a user list screen generated based on the user list 234. The user list screen may be, as an example, the user list screen 731 shown in FIG. 18(B). The user list screen 731 consists of record images corresponding to each record in the user list 234. In the example of FIG. 18(B), record images 732A to 732C are described as record images, but the number of record images is not limited to three. In the example of FIG. 18(B), when the number of records in the user list 234 is more than three (that is, the number of users participating in the game is more than three), the player can cause the display unit 452 to display other record images by inputting an operation of scrolling the screen (for example, a drag operation or a flick operation) on the touch screen 45.

[0184] As an example, the record images 732A to 732C each include a user name 733A to 733C, tag information 734A to 734C, and icons 735A to 735C. Hereinafter, when there is no need to distinguish between the record images 732A to 732C, the user names 733A to 733C, the tag information 734A to 734C, and the icons 735A to 735C, they are described as "record image 732", "user name 733", "tag information 734", and "icon 735", respectively.

[0185] The user name 733 is information indicating each user who participated in the game and is stored in the "User" column in the user list 234. The tag information 734 is information indicating the tags associated with each piece of information indicating each user who participated in the game in the user list 234. For example, the record image 732A includes "AAAAA" as the user name 733A. Therefore, the record image 732A includes, as the tag information 734A, "Magazine, 10F, Boss, 'Win against the Boss with a Magazine Present'" associated with "AAAAA" in the user list 234. The icon 735 is, for example, an image pre-set by the user.

[0186] Note that the processor 40 may store the received user list in the distribution terminal 400 (user list 421 in FIG. 7). The download screen 721 may include a UI image (not shown) for displaying the user list 421 on the display unit 452. In this example, when the UI image is tapped, the processor 40 does not download the user list 234, but reads out the user list 421, generates a user list screen from the user list 421, and causes it to be displayed on the display unit 452.

[0187] In step S64, the processor 40, as the operation reception unit 413, receives a second operation for selecting any one of the users included in the user list screen 731. The second operation may be, as an example, an operation of tapping any one of the record images 732 on the user list screen 731. In the example of FIG. 18(B), the player inputs a tap operation on the record image 732A. That is, the player selects the user "AAAAA" as the user who distributes the operation instruction data.

[0188] When a tap operation on the record image 732 is received, in step S65, the processor 40 causes the display control unit 412 to display the motion list 422 on the display unit 452. Specifically, the processor 40 causes the display unit 452 to display a motion list screen generated based on the motion list 422. The motion list screen may be, for example, the motion list screen 741 shown in FIG. 19. The motion list screen 741 consists of record images corresponding to the respective records in the motion list 422. In the example of FIG. 19, record images 742A to 742C are described as the record images, but the number of record images is not limited to three. In the example of FIG. 19, when the number of records in the motion list 422 is more than four, the player can cause the display unit 452 to display other record images by inputting an operation for scrolling the screen (for example, a drag operation or a flick operation) to the touch screen 45.

[0189] As an example, the record images 742A to 742C each include a motion name 743A to 743C, a motion image 744A to 744C, and a UI image 745A to 745C. Hereinafter, when there is no need to distinguish between the record images 742A to 742C, the motion names 743A to 743C, the motion images 744A to 744C, and the UI images 745A to 745C, they will be described as "record image 7432", "motion name 743", "motion image 744", and "UI image 745", respectively.

[0190] Motion name 743 is information for identifying the motion stored in the motion list 422. Motion image 744 is an image generated from the motion data associated with each motion name in the motion list 422. The processor 40, as an example, includes, as the motion image 744 and the record image 742, an image of the avatar object 610 taking the first posture in each motion data. The motion image 744 may be a UI image that accepts a predetermined operation by the player (for example, a tap operation on the motion image 744). When receiving the predetermined operation, the processor 40 may play a motion video in which the avatar object 610 operates based on the motion data. When the motion video ends, the processor 40 may automatically redisplay the motion list screen 741.

[0191] Note that, instead of the motion image 744, the record image 742 may include, for example, a UI image including text such as "Play Motion".

[0192] In step S66, the processor 40, as the operation reception unit 413, receives a third operation for selecting a motion. The third operation may be, as an example, a tap operation on the UI image 745. That is, the UI image 745 receives an operation for selecting the motion data corresponding to each record image 742. By receiving the third operation, the processor 40, as the motion identification unit 415, identifies the motion data selected by the player.

[0193] In step S67, the processor 40, as the display control unit 412 and the voice reception unit 414, receives the voice input of the player while playing a motion video in which the avatar object 610 operates based on the selected motion data.

[0194] FIG. 20 is a diagram showing a specific example of voice input by player 4. As shown in FIG. 20, while playing motion video 810A, player 4 is inputting uttered voice 820A. This uttered voice 820A is an uttered voice addressed to user 3 (hereinafter, user 3A) whose user name is "AAAAA". That is, in the example of FIG. 20, player 4 selects user 3A (the first user) in step S64 and creates operation instruction data addressed to the user 3A. It is assumed that the user terminal 100 used by user 3A is user terminal 100A.

[0195] Since the uttered voice 820A is an uttered voice addressed to user 3A, it is an uttered voice based on the content of the support provided by the user 3A to the avatar object 610 (in other words, player 4). Specifically, in the boss battle at stage 10F, user 3A inserted a magazine, and avatar object 610 won against the boss with the bullets in the inserted magazine. For this reason, the uttered voice 820A has the content of "Thank you for presenting the magazine in the boss battle! The timing was perfect too! Thanks to Mr. AAAAA, I was able to clear it!". Thus, it is preferable that the uttered voice includes the content of the support provided by user 3 in the game and the gratitude to user 3.

[0196] In a certain situation, before starting voice input, that is, before inputting the third operation to distribution terminal 400, player 4 creates the uttered content addressed to user 3. In another situation, the uttered content addressed to user 3 may be automatically generated by processor 40. Also, processor 40 may superimpose and display the tag associated with user 3 selected by the second operation on motion video 810A.

[0197] Processor 40 converts the received voice into voice data. In step S68, processor 40, as operation instruction data generation unit 416, generates operation instruction data including the voice data and the motion data of the selected motion.

[0198] In step S69, the processor 40, acting as the communication control unit 411, distributes the generated operation instruction data to the user terminal 100 (first computer) of the selected user 3 (user 3A in the example of FIG. 20). FIG. 21 is a diagram showing yet another specific example of the screen displayed on the distribution terminal 400. After executing step S68, the processor 40, acting as the display control unit 412, causes the distribution screen to be displayed on the display unit 452. The distribution screen may be, for example, the distribution screen 751 shown in FIG. 21(A). The distribution screen 751 includes a UI image 752 and a motion image 753A. Also, the distribution screen 751 may include information indicating the user who is the destination of the distribution of the operation instruction data, as shown in FIG. 21(A).

[0199] The UI image 752 accepts an operation for distributing the operation instruction data to the selected user 3. The operation may be, for example, a tap operation on the UI image 752. The motion image 753A is a UI image that accepts an operation for playing a video based on the generated operation instruction data, that is, a video based on the operation instruction data generated for user 3A. The operation may be, for example, a tap operation on the motion image 753A. Note that the UI image that accepts an operation for playing the generated video is not limited to the motion image 753A. For example, it may be a UI image including text such as "Play Video". The processor 40 may automatically redisplay the distribution screen 751 when the video ends.

[0200] The distribution screen 751 preferably further includes a UI image that accepts an operation for returning to accepting voice input. The operation may be, for example, a tap operation on the UI image. By including the UI image in the distribution screen 751, the player 4 can perform voice input again, for example, when the voice input fails, such as when the content spoken is incorrect. Note that the UI image may be a UI image that accepts an operation for returning to selecting motion data.

[0201] When the processor 40 receives a tap operation on the UI image 752, it transmits operation instruction data to the server 200 together with information indicating the user 3A. Based on the information indicating the user 3A, the server 200 identifies the user terminal 100 that is the destination of the operation instruction data and transmits the operation instruction data to the identified user terminal 100 (i.e., the user terminal 100A).

[0202] When the transmission of the operation instruction data is completed, as an example, the processor 40 may cause the display unit 452 to display a delivery completion screen 761 shown in FIG. 21(B). The delivery completion screen 761 includes, as an example, UI images 762 and 763. Further, the delivery completion screen 761 may include text indicating that the transmission of the operation instruction data has been completed, as shown in FIG. 21(B).

[0203] The UI image 762 receives an operation for starting the creation of operation instruction data for another user 3. The operation may be, for example, an operation of tapping the UI image 762. When the processor 40 receives the tap operation, it causes the user list screen to be displayed again on the display unit 452. That is, when the tap operation is received, the delivery process returns to step S63. At this time, the processor 40 may generate a user list screen based on the user list 421 stored in the delivery terminal 400 and cause the display unit 452 to display it. The UI image 763 receives an operation for ending the application. The operation may be, for example, an operation of tapping the UI image 763. When the operation is received, the delivery process ends.

[0204] In the example described with reference to FIGS. 20 and 21, as shown in FIG. 21(C), the delivery terminal 400 transmits the operation instruction data of the video for the user 3A (the user 3 with the user name "AAAAA") only to the user terminal 100A.

[0205] FIG. 22 is a diagram showing another specific example of voice input by player 4. As shown in FIG. 22, while player 4 is playing motion video 810B, it is inputting uttered voice 820B. This uttered voice 820B is an uttered voice addressed to user 3 (hereinafter, user 3B) whose user name is "BBBBB". That is, in the example of FIG. 22, player 4 inputs a tap operation on record image 732B corresponding to user 3B in step S64 and creates operation instruction data addressed to user 3B. It is assumed that the user terminal 100 used by user 3B is user terminal 100B.

[0206] Since uttered voice 820B is an uttered voice addressed to user 3B, it is an uttered voice based on the content of the support given by the user 3B to avatar object 610 (in other words, player 4). Specifically, in the battle between user 3B and the minion enemy at stage 3F, user "BBBBB" throws a first aid kit, and as a result, the physical strength of avatar object 610 is about to reach 0 (game over) but the physical strength recovers. For this reason, the content of uttered voice 820B is "Thanks to the first aid kit given by Mr. BBBBB, I didn't game over at 3F. Thank you so much!"

[0207] FIG. 23 is a diagram showing yet another specific example of the screen displayed on distribution terminal 400. The distribution screen 751 shown in FIG. 23(A) includes UI image 752 and motion image 753B. When motion image 753B receives a tap operation, it plays a video based on the operation instruction data generated for user 3B.

[0208] When a tap operation on UI image 752 is received, processor 40 transmits the operation instruction data to server 200 together with the information indicating user 3B. Server 200 identifies user terminal 100, the transmission destination of the operation instruction data, based on the information indicating user 3B, and transmits the operation instruction data to the identified user terminal 100 (that is, user terminal 100B).

[0209] In the example described with reference to FIGS. 22 and 23, as shown in FIG. 23(C), the distribution terminal 400 transmits the operation instruction data of the video addressed to user 3B (user 3 with the user name "BBBBB") only to the user terminal 100B.

[0210] As described above, the content of the voice based on the voice data included in the operation instruction data is based on the content of the support provided by user 3 to player 4 in the most recent game participation. Since the content of the support varies for each user 3, the content of the voice also varies for each user 3. That is, after the game ends, operation instruction data including voices of different contents is transmitted to at least some of the user terminals 100 of the users 3 who participated in the game.

[0211] Also, the motion of the avatar object 610 in the example of FIG. 22 is different from the motion in the example of FIG. 20. That is, player 4 selects different motion data in generating the operation instruction data addressed to user 3B than when generating the operation instruction data addressed to user 3A. Specifically, in step S66, player 4 inputs a tap operation on the UI image 745B to select the motion data corresponding to the record image 742B. In this way, player 4 can make the motion data included in the operation instruction data different for each user 3.

[0212] The operation instruction data for each user 3, which includes voice data of different contents for each user 3 and the motion data selected for each user 3, is transmitted only to the user terminal 100 of each user 3. In other words, unique (one-of-a-kind) operation instruction data is transmitted to each of the user terminals 100 of the selected users 3.

[0213] FIG. 24 is a diagram showing an overview of the transmission of game progress information from the game play terminal 300 to the user terminal 100. While the operation instruction data for video playback in the user terminal 100 is unique for each user terminal 100, as shown in FIG. 24, during game execution, the game progress information transmitted to the user terminals 100 of all users 3 participating in the game is common among the respective user terminals 100. That is, the operation instruction data included in the game progress information is also common among the respective user terminals 100. Thus, it can be said that the operation instruction data for video playback and the operation instruction data for advancing the game are different data in terms of differences between user terminals 100 and the destination of transmission.

[0214] (Video playback process in user terminal 100) FIG. 25 is a flowchart showing an example of the flow of the video playback process executed in the user terminal 100.

[0215] In step S71, the processor 10 receives the operation instruction data as the video playback unit 117. In step S72, the processor 10 notifies user 3 of the reception of the operation instruction data as the video playback unit 117. As an example, the processor 10 notifies user 3 of the reception of the operation instruction data by at least any one of the display on the display unit 152 of the notification image, the reproduction of the notification sound from a speaker (not shown), and the lighting or flashing of a lighting unit (not shown) composed of an LED (light-emitting diode) or the like.

[0216] In step S73, the processor 10 receives, as the operation reception unit 111, a first playback operation for playing a video. As an example, the first playback operation may be an operation of tapping a notification image. In step S74, the processor 10, as the video playback unit 117, renders the operation instruction data and plays the video. As an example, the processor 10 may start an application for playing this game and play the video, or may start a separate application for video playback and play the video. Hereinafter, the video will be referred to as the "thank you video".

[0217] FIG. 26 is a diagram showing a specific example of playing the thank you video. Specifically, it is a diagram showing an example of playing the thank you video on the user terminal 100 of the user 3A. In the thank you video 910A played on the user terminal 100, the avatar object 610 is speaking the voice 920A while performing a certain motion. In other words, the processor 10 outputs the voice 920A from a speaker (not shown) while playing the thank you video 910A including the avatar object 610 that performs a certain motion.

[0218] The motion in the thank you video 910A is based on the motion data selected by the player 4 in the generation of the operation instruction data for the user 3A, and the voice 920A is based on the voice data generated from the spoken voice 820A input by the player 4 in the generation of the operation instruction data. That is, the voice 920A is a voice including the content of the support provided by the user 3A in the game and the gratitude for the support. In this way, by inputting the first playback operation, the user 3A can watch the thank you video in which the avatar object 610 speaks the content of the support provided by the user 3A in the game and the gratitude for the support.

[0219] As an example, after the playback of the thank-you video 910A ends, the user terminal 100 may display at least one UI image on the touch screen 15. The UI image may be, for example, a UI image that accepts an operation to play the thank-you video 910A again, or a UI image that accepts an operation to transition to another screen, or a UI image that accepts an operation to end the application.

[0220] Also, as an example, during the playback of the thank-you video 910A, the user terminal 100 may display at least one UI image on the touch screen 15. The UI image may be, for example, a plurality of UI images that respectively accept operations to temporarily stop, end, or change the playback scene of the thank-you video 910A being played.

[0221] Note that these UI images displayed during the playback of the thank-you video 910A and after the playback of the thank-you video 910A ends do not include a UI image for responding to the avatar object 610. That is, in the thank-you video 910A according to the present embodiment, there is no means for responding to the avatar object 610.

[0222] FIG. 27 is a diagram showing another specific example of the playback of the thank-you video. Specifically, it is a diagram showing an example of the playback of the thank-you video on the user terminal 100 of the user 3B. In the thank-you video 910B played on the user terminal 100, the avatar object 610 is speaking the voice 920B while performing a certain motion. In other words, the processor 10 outputs the voice 920B from a speaker (not shown) while playing the thank-you video 910B including the avatar object 610 that performs a certain motion.

[0223] The motion in the thank-you video 910B is based on the motion data selected by the player 4 in generating the operation instruction data addressed to the user 3B. The voice 920B is based on the voice data generated from the uttered voice 820B input by the player 4 in generating the operation instruction data. Therefore, in the example of FIG. 27, the motion performed by the avatar object 610 is different from the motion in the example of FIG. 26. Also, the voice 920B is a voice that includes the content of the assistance provided by the user 3B in the game and the gratitude for the assistance. Therefore, in the example of FIG. 27, the content of the voice 920B is different from the content of the voice 920A in the example of FIG. 26.

[0224] In this way, after the game ends, the thank-you video received by at least some of the user terminals 100 of the users 3 who participated in the game is a video in which the uttered content of the avatar object 610 is different for each user 3.

[0225] Note that the processor 10 may superimpose and display a UI image 930 including content that encourages participation in the next game on the video 910. The UI image 930 may be distributed together with the operation instruction data, or may be stored in the user terminal 100 as game information 132.

[0226] In the live distribution game of this embodiment, operation instruction data for making an image within the virtual space 600B in which an object including the avatar object 610 is arranged the display image is live-distributed from the server 200 to a plurality of user terminals (viewing terminals) 100. Each of the plurality of user terminals 100 displays an image within the virtual space 600B on the touch screen 15 based on the live-distributed operation instruction data (see FIG. 29). When a touch operation is performed on an object arranged in the virtual space 600B, the user terminal 100 executes control according to the type of the object. Hereinafter, the specific content of the live distribution game will be described.

[0227] In a live streaming game, the same virtual space 600B as the virtual space 600A defined by the game play terminal 300 is defined by the user terminal 100. The game play terminal 300 distributes game progress information including operation instruction data capable of specifying the movement and speech voice of the performer to a plurality of user terminals 100 via the server 200. In the memory 11 of the user terminal 100, a plurality of types of object data for displaying an object including the avatar object 610 are stored in advance. Also, an object ID for specifying the type of the object is stored in the memory 11.

[0228] When an operation for starting the live streaming game is performed, the user terminal 100 specifies the operation instruction data from the game progress information distributed from the game play terminal 300 via the server 200, and analyzes (renders) the operation instruction data to specify the type, position, posture, etc. of the object. Further, the user terminal 100 arranges an object including the avatar object 610 in the virtual space 600B based on the analysis result, the object data stored in the memory 11, and the object ID.

[0229] On the monitor 51 of the HMD 500, a view image 650 of a view area 640A corresponding to the position, orientation, and tilt of the virtual camera 620A arranged in the virtual space 600A is displayed. Also, on the touch screen 15 of the user terminal 100, a view image 660 of a view area 640B corresponding to the position, orientation, and tilt of the virtual camera 620B arranged in the virtual space 600B is displayed. From a speaker (not shown) provided in the user terminal 100, the speech voice of the performer is output based on the operation instruction data specified from the game progress information.

[0230] The touch operation on the object arranged in the virtual space 600B is specifically a touch operation on the object displayed on the touch screen 15 (i.e., the object appearing in the view image 660 of the virtual camera 620B). On the other hand, the position, posture, etc. of the virtual camera 620B, that is, the viewpoint capturing the virtual space 600B, can be changed according to the user's operation. For this reason, the user terminal 100 specifies the viewpoint capturing the virtual space 600B and the touch position on the touch screen 15, and specifies the object touched based on the viewpoint and the touch position. Further, if the specified object is the avatar object 610, the user terminal 100 specifies the touch part (touch location) on the avatar object 610 based on the touch position.

[0231] The control according to the type of the touched object includes control for inserting a coin item, control for eliciting a reaction from the performer according to a body touch on the avatar object 610, control for changing the clothing of the avatar object 610, and control for executing a reaction according to a touch on an object other than the avatar object 610.

[0232] (Control for inserting a coin item) FIG. 28(A) is a diagram showing an example of a coin item management table TBL1 in which coin items that can be inserted by the user (reflected in the virtual space 600B) and the amount of coins (i.e., the charging amount) charged by the insertion of the coin item are registered. The table TBL1 is stored in the memory 21 of the server 200 and is updated at any time. Further, the table TBL1 is transmitted to the user terminal 100 by the server 200 when the user terminal 100 logs in to the live distribution game, and is stored in the memory 11 of the user terminal 100.

[0233] According to FIG. 28(A), as insertable coin items, a star-shaped cushion, a heart-shaped cushion, a foliage plant, a round table, a piano, etc. are provided. For each of the coin items of the star-shaped cushion and the heart-shaped cushion, 200 coins are associated as the amount of coins to be charged when inserting the coin item. For the coin item of the foliage plant, 300 coins are associated as the amount of coins to be charged when inserting the coin item. For the coin item of the round table, 500 coins are associated as the amount of coins to be charged when inserting the coin item. For the coin item of the piano, 800 coins are associated as the amount of coins to be charged when inserting the coin item.

[0234] Referring to FIG. 29(A), each of the virtual spaces 600A and 600B is a space imitating a room in a high-rise apartment. On the floor object 685, a plush toy object 684, an object imitating a bed, an object imitating a low table, an object imitating lighting (not shown), etc. are arranged. Also, on the floor object 685, as an object of a coin item inserted by a user (for example, another user operating another user terminal 100) who is watching a live stream, a foliage plant object (hereinafter referred to as "foliage plant object") 675A is arranged. Further, on the wall, an object imitating a lighting switch (hereinafter referred to as "lighting switch object") 683 is arranged. Further, in the virtual space 600B, an avatar object 610 that operates in accordance with the movement of the performer is arranged. On the touch screen 15 of the user terminal 100, a view image 660 captured by the virtual camera 620B among the images in the virtual space 600B is displayed.

[0235] Here, when a touch operation is performed on the object of the coin-throwing item (for example, the foliage plant object 675A) arranged in the virtual space 600B on the touch screen 15, the user terminal 100 identifies the type of the object based on the operation instruction data, and identifies the coin-throwing item corresponding to the type and the charging amount of the coin-throwing item from the coin-throwing item management table TBL1. Further, the user terminal 100 displays an inquiry screen 681 (the first screen, the first UI (User Interface)) for inquiring about the purchase of the coin-throwing item by superimposing it on the view image 660 (see FIG. 29(B)). According to FIG. 29(B), the name of the coin-throwing item (for example, foliage plant) and the charging amount (for example, 300 coins) are displayed on the inquiry screen 681, and further, a "purchase" icon and a "cancel" icon are displayed.

[0236] When a touch operation is performed on the "purchase" icon, the user terminal 100 transmits charging information including the name of the purchased coin-throwing item and the user name to the server 200 in order to request the server 200 to perform a charging process. The server 200 identifies the charging amount from the coin-throwing item management table TBL1 based on the charging information, and charges the identified charging amount to the user identified based on the charging information. For example, if the name of the coin-throwing item and the user name are foliage plant and AAA respectively, the server 200 charges 300 coins to the user AAA.

[0237] After the charging process, the user terminal 100 transmits item input information including the name of the coin item and the user name to the game play terminal 300 via the server 200, and makes the inquiry screen 681 invisible. The game play terminal 300 identifies the name of the coin item from the item input information, places an object of the name in the virtual space 600A, and distributes game progress information including operation instruction data corresponding to the object to a plurality of user terminals 100. As a result, the object of the coin item is placed in the virtual space 600B defined by each of the plurality of user terminals 100. Therefore, when the "Purchase" icon is touched on the inquiry screen 681 shown in FIG. 28(B), foliage plant objects 675B of the same type as the foliage plant object 675A are placed in each of the virtual spaces 600A and 600B. This makes it possible to promote the interaction between the user and the performer.

[0238] (Control for eliciting a reaction from the performer according to body touch) Referring to FIG. 29(C), when a touch operation on the avatar object 610 is performed on the touch screen 15, the user terminal 100 identifies the touch part on the avatar object 610 based on the touch position, and transmits touch part information for enabling the performer to identify the touch part to the game play terminal 300 via the server 200. The touch part information includes the touch part and the user name.

[0239] At the position in front of the performer in the virtual space 600A, a mirror image corresponding to a mirror that projects the avatar object 610 is displayed. The performer can confirm how the avatar object 610 is displayed on the user terminal 100 by the mirror image. The game play terminal 300 identifies the touch part and the user name from the touch part information, notifies the touch part on the avatar object 610 in the mirror image (for example, makes it shine brightly), and displays the user name on the monitor 51 of the HMD 500.

[0240] As a result, the performer can recognize which part has been touched by which user, and can react in real time by moving the body or speaking. As a result, diversification of interaction between the user and the performer can be achieved.

[0241] (Control for changing clothes) FIG. 28(B) is a diagram showing an example of a clothing management table TBL2 in which the clothing of the avatar object 610 that can be changed by the user and the amount of coins charged for the change are registered. The table TBL2 is stored in the memory 21 of the server 200 and is updated at any time. Further, the table TBL2 is transmitted to the user terminal 100 by the server 200 when the user terminal 100 logs in to the live distribution game, and is stored in the memory 11 of the user terminal 100.

[0242] According to FIG. 28(B), as clothing, T-shirts, sweaters, jeans, miniskirts, Santa costumes, etc. are provided. For the T-shirt, 200 coins are associated as the amount of coins charged at the time of replacement. For the sweater, 300 coins are associated as the amount of coins charged at the time of replacement. For the jeans, 500 coins are associated as the amount of coins charged at the time of replacement. For the miniskirt, 700 coins are associated as the amount of coins charged at the time of replacement. For the Santa costume, 1000 coins are associated as the amount of coins charged at the time of replacement.

[0243] When a touch operation is performed on a part of the avatar object 610 wearing clothing, the user terminal 100 identifies, from the clothing management table TBL2, candidates for clothing to be worn by the avatar object 610 in place of the current clothing and the charging amounts for the candidates, and superimposes and displays an inquiry screen 682 (second screen, second UI) for inquiring about the purchase of clothing on the view image 660 (see Fig. 29(D)). The names of the candidate clothing are listed at the left position on the inquiry screen 682, and the charging amounts are displayed at the right position of the names. Further, a "Purchase" icon is displayed on the right side of the charging amount, and a "Cancel" icon is displayed at the lower right position on the inquiry screen.

[0244] For example, if the touch part is the chest of the avatar object 610, a T-shirt, a sweater, a Santa costume, their respective charging amounts of 200 coins, 300 coins, 1000 coins, a "Purchase" icon, and a "Cancel" icon are displayed on the inquiry screen 682. Also, if the touch part is the lower part of the waist of the avatar object 610, jeans, a miniskirt, their respective charging amounts of 500 coins, 700 coins, a "Purchase" icon, and a "Cancel" icon are displayed on the inquiry screen 682.

[0245] When a touch operation is performed on the "Purchase" icon, the user terminal 100 transmits charging information including the name of the purchased clothing and the user name to the server 200 for charging processing. The server 200 identifies the charging amount from the clothing management table TBL2 based on the charging information and charges the identified user the identified charging amount. For example, if the name of the clothing and the user name are Santa costume and AAA respectively, the server 200 charges user AAA 1000 coins.

[0246] After the charging process, the user terminal 100 identifies the clothing associated with the touched part among the clothing of the avatar object 610 placed in its virtual space 600B as the clothing to be changed, and changes the clothing to the purchased clothing. Specifically, the object (texture image) of the clothing to be changed is switched to the object (texture image) of the purchased clothing. Note that the clothing to be changed is specified according to the purchased clothing.

[0247] Therefore, if the part touched by the user is the chest of the avatar object 610 and the clothing purchased by the user is Santa's clothing, the clothing worn by the avatar object 610 all over the body is changed to the Santa's clothing (see Fig. 29(E)). Also, if the part touched by the user is the lower part of the waist of the avatar object 610 and the clothing purchased by the user is a mini skirt, the short pants and long tights worn by the avatar object 610 are changed to the mini skirt.

[0248] At this time, the clothing of the avatar object 610 displayed on the user terminal 100 of other users cannot be changed. Therefore, each of the plurality of users watching the live distribution can enjoy the live distribution while seeing the avatar object 610 wearing their favorite clothing.

[0249] (Control for executing a reaction according to a touch on another object) Fig. 28(C) is a diagram showing an example of a reaction management table TBL3 in which objects capable of generating a reaction by a touch operation and the content of the reaction generated by the touch operation are registered. The table TBL3 is stored in the memory 21 of the server 200 and is updated at any time. Also, the table TBL3 is transmitted to the user terminal 100 by the server 200 when the user terminal 100 logs in to the live distribution game and is stored in the memory 11 of the user terminal 100.

[0250] According to FIG. 28(C), as reaction-capable objects, a floor object, a lighting switch object, a plush object, etc. are provided. For the floor object, changing the color of the floor is associated as the content of the reaction. For the lighting switch object, changing the brightness of the lighting is associated as the content of the reaction. For the plush object, generating a predetermined sound effect (for example, a crying sound) is associated as the content of the reaction.

[0251] When a touch operation is performed on an object other than the avatar object 610 arranged in the virtual space 600B on the touch screen 15, the user terminal 100 identifies the type of the object based on the operation instruction data, and identifies the content of the reaction corresponding to the type from the reaction management table TBL3. Further, the user terminal 100 causes a change in its own virtual space 600B or generates a sound effect from the speaker it has, based on the identified content of the reaction.

[0252] Therefore, when the floor object 685 shown in FIG. 29(A) is touched, the color of the floor object 685 changes. Also, when the lighting switch object 683 is touched, the brightness of the lighting arranged in the room (that is, the brightness of the room) changes. Further, when the plush object 684 is touched, a sound effect is generated from the speaker. Thereby, the entertainment value of the live distribution game is improved.

[0253] Note that the user can also insert a coin item by tapping the UI image 701 and then tapping the UI images 711A to 711D within the UI image 711 thereby displayed. However, the coin items that can be inserted in the live distribution shown in FIG. 29 are the coin items registered in the coin item management table TBL1 shown in FIG. 28(A). When a coin item is inserted, the object of the coin item is arranged in each of the virtual spaces 600A and 600B.

[0254] In addition, the user can also send comments to the performer to the game play terminal 300 by tapping the UI image 702 and entering comments. The game play terminal 300 displays the comment on the monitor 51 of the HMD 500 and distributes it to a plurality of user terminals 100. As a result, the comment is superimposed and displayed on the view images 660 of the plurality of user terminals 100.

[0255] (Regarding the operation) FIG. 30 is a flowchart showing an example of the flow of reaction control processing executed by the user terminal 100.

[0256] In step S81, the processor 10 of the user terminal 100 determines whether a touch operation has been performed based on an input operation on the touch screen 15. If it is determined that the touch operation has not been performed, the process returns. If it is determined that the touch operation has been performed, the process proceeds to step S82. In step S82, it is determined whether the target of the touch operation is an object within the virtual space 600B based on the position of the object in the virtual space 600B, the viewpoint capturing the virtual space 600B, and the touch position on the touch screen 15. When the inquiry screen 681 or 682 is superimposed and displayed on the view image 660, in step S82, it is not determined that the target of the touch operation is an object within the virtual space 600B.

[0257] When it is determined that the object of the touch operation is an object within the virtual space 600B, the process proceeds to step S83, and it is determined based on the operation instruction data whether the object is an object of a coin - throwing item. When it is determined that the object is an object of the coin - throwing item, the process proceeds to step S84. In step S84, the name and the charging amount of the coin - throwing item are specified based on the coin - throwing item management table TBL1, and then the process proceeds to step S85. When proceeding from step S84 to step S85, in step S85, an inquiry screen 681 including the name and the charging amount specified in step S84 is superimposed on the view image 660 and displayed. When the display is completed, the process returns.

[0258] In step S83, when it is not determined that the object of the touch operation is an object of a coin - throwing item, the process proceeds to step S86. In step S86, it is determined based on the operation instruction data whether the object of the touch operation is the avatar object 610. When it is determined that the object is the avatar object 610, the process proceeds to step S87. In step S87, the touch part on the avatar object 610 is specified, and touch part information including the touch part and the user name is transmitted to the game play terminal 300 via the server 200.

[0259] In step S88, it is determined based on the position of the object in the virtual space 600B, the viewpoint capturing the virtual space 600B, and the touch position on the touch screen 15 whether the touch position is a part where the avatar object 610 is wearing clothing. When it is not determined that the touch part is a part where clothing is worn, the process returns. On the other hand, when it is determined that the touch part is a part where clothing is worn, the process proceeds to step S89. In step S89, the name of the changeable clothing and the charging amount of the clothing are specified from the clothing management table TBL2, and then the process proceeds to step S85. When proceeding from step S89 to step S85, in step S85, an inquiry screen 682 including the name and the charging amount specified in step S84 is superimposed on the view image 660 and displayed. When the display is completed, the process returns.

[0260] In step S86, when it is not determined that the target of the touch operation is the avatar object 610, the process proceeds to step S90. In step S90, it is determined based on the operation instruction data and the reaction management table TBL3 whether the target of the touch operation is an object capable of reacting. When it is not determined that the object is an object capable of reacting, the process returns. On the other hand, when it is determined that the object is an object capable of reacting, the process proceeds to step S91. In step S91, the content of the reaction corresponding to the object capable of reacting is specified from the reaction management table TBL3, and based on the specified content of the reaction, a change is caused in its own virtual space 600B or a sound effect is generated from its own speaker. When the process of step S91 is completed, the process returns.

[0261] In step S82, when it is not determined that the target of the touch operation is an object within the virtual space 600B, the process proceeds to step S92, and it is determined based on the processing result of step S85 whether the inquiry screen 681 or 682 is being displayed. When it is determined that the inquiry screen 681 or 682 is being displayed, the process proceeds to step S93, and it is determined based on the input operation on the touch screen 15 whether the "Purchase" icon has been touched. When it is determined that the "Purchase" icon has been touched, in order to request the server 200 to perform the charging process, the charging information including the name of the purchased offering item or clothing and the user name is transmitted to the server 200. When the charging process is completed, the process proceeds to step S95.

[0262] In step S95, it is determined based on the processing result of step S85 whether the purchase target is a coin item. If it is determined that the item is a coin item, the process proceeds to step S96, and item input information including the name of the purchased coin item and the user name is transmitted to the game play terminal 300 via the server 200. On the other hand, in step S95, when it is determined that the purchase target is not a coin item, that is, when it is determined that the purchase target is clothing, the process proceeds to step S97. In step S97, the clothing of the avatar object 610 arranged in its own virtual space 600B is changed to the purchased clothing. When the processing of step S96 or S98 is completed, the process proceeds to step S98, and the displayed inquiry screen 681 or 682 is made non-displayed. When the processing of step S98 is completed, the process returns.

[0263] When it is determined in step S93 that the "Purchase" icon has not been touched, the process proceeds to step S99. In step S99, it is determined based on the input operation on the touch screen 15 whether the "Cancel" icon has been touched. When it is determined that the "Cancel" icon has not been touched, the displayed inquiry screen 681 or 682 in step S98 is made non-displayed, and then the process returns. Also, in step S92, when it is determined that the inquiry screen 681 or 682 is not being displayed, other processing is executed in step S100, and then the process returns.

[0264] <Effects of the present embodiment> According to the present embodiment, operation instruction data for enabling the display of an image in the virtual space 600 in which an object including the avatar object 610 is arranged is live-distributed by the server 200. On the touch screen 15 of the user terminal 100, an image in the virtual space 600B is displayed based on the operation instruction data. When a touch operation is performed on an object arranged in the virtual space 600B, the user terminal 100 executes control according to the type of the object. Thereby, the modes of reaction to the user's operations are diversified, and the enjoyment of the live-distribution game is improved.

[0265] Further, according to the present embodiment, if the touched object is an object of a coin-throwing item, an object of the same type as the touched object is reflected in each of the virtual spaces 600A and 600B. This makes it possible to promote the interaction between the user and the performer.

[0266] Furthermore, according to the present embodiment, a query screen 681 asking whether to purchase or not is displayed on the touch screen 15 according to the touched object. Therefore, it is possible to avoid the inconvenience that the charging process is executed by accidentally touching the object of the coin-throwing item arranged in the virtual space 600B.

[0267] Also, according to the present embodiment, the charging amount of the coin-throwing item corresponding to the touched object is displayed on the query screen 681. Thereby, the user can carefully judge whether to purchase the coin-throwing item.

[0268] Furthermore, according to the present embodiment, if the touched object is the avatar object 610, the object of the clothing associated with the touched part of the avatar object 610 is switched to an object of another clothing. Thereby, the user can enjoy the live distribution while seeing the avatar object 610 wearing the clothing of his / her preference.

[0269] Also, according to the present embodiment, a query screen 682 asking whether to purchase or not is displayed on the touch screen 15 according to the object of the other clothing. Therefore, it is possible to avoid the inconvenience that the charging process is executed by accidentally touching the clothing of the avatar object 610.

[0270] Furthermore, according to the present embodiment, the charging amount of the other clothing is displayed on the query screen 682. Thereby, the user can carefully judge whether to purchase the other clothing.

[0271] Also, according to the present embodiment, in response to a touch operation on the avatar object 610, touch position information for enabling the performer to identify the touched part is transmitted to the game play terminal 300 via the server 200. Thereby, the performer can recognize which part has been touched, and can react in real time by moving the body or speaking.

[0272] Furthermore, according to the present embodiment, based on the viewpoint capturing the virtual space 600B and the touch position on the touch screen 15, the object targeted by the touch operation is identified. Thereby, even when the viewpoint capturing the virtual space 600B changes, the object targeted by the touch operation can be accurately identified.

[0273] <Modification Example> Modification examples of the embodiments described above are listed below.

[0274] (1) In the above embodiment, the clothing of the replaceable avatar object 610 is set to be purchased at the time of replacement. However, the clothing may be purchasable in advance. In this case, when the clothing of the avatar object 610 is touched, it may be replaced with the clothing purchased in advance without displaying the inquiry screen 682. Also, the unpurchased clothing and the purchased clothing may be displayed on the inquiry screen 682, and when the purchased clothing is selected, the replacement may be performed without performing the charging process.

[0275] (2) In the above embodiment, the coin-throwing item is also set to be purchased at the time of insertion. However, it may also be possible to purchase the coin-throwing item in advance. In this case, objects of the same type as the object of the coin-throwing item are arranged in the virtual space 600B, and when an object of the same type is touched, the coin-throwing item may be inserted without displaying the inquiry screen 681. Note that the coin-throwing items for which advance purchase is appropriate include items sold for a limited period.

[0276] (3) In the above embodiment, the coin-throwing item can be inserted (reflected in the virtual space 600B) by purchasing it. However, if a location information game is associated with the live distribution game and a region-specific coin-throwing item, i.e., a local item, is obtained in the location information game, the local item may be made insertable.

[0277] (4) In the above embodiment, the contents of each of the coin-throwing item management table TBL1, clothing management table TBL2, and reaction management table TBL3 transmitted from the server 200 at the time of login are common among users. However, the contents may be made to differ according to the user's intimacy with the avatar object 610. For example, for users with a high intimacy level, the number of types of coin-throwing items registered in the coin-throwing item management table TBL1 may be made larger than that for users with a low intimacy level, the degree of exposure of the clothing registered in the clothing management table TBL2 may be made higher than that for users with a low intimacy level, and the number of types of objects that can react registered in the reaction management table TBL3 may be made larger than that for users with a low intimacy level.

[0278] (5) In the above embodiment, any user can perform the dressing operation of changing the clothing of the avatar object 610. However, the users who can perform the dressing operation may be specified according to the user's intimacy with the avatar object 610 or anniversaries such as the user's birthday.

[0279] (6) In the above-described embodiment, by touching the object of the offering money item arranged in the virtual space 600B, an object of the same type as the said object is arranged in the said virtual space 600B. However, the object of the same type may have different colors, sizes, etc. from the touched object.

[0280] (7) In the above-described embodiment, the clothing of the avatar object 610 can be changed to another clothing. However, instead of or together with the clothing, it may be possible to change the accessories worn by the avatar object 610.

[0281] (8) In the above-described embodiment, an example in which the inquiry screen in step S85 is displayed at a predetermined position substantially in the center of the display area of the touch screen 15 has been described. For this reason, the inquiry screen may overlap the avatar object 610. As shown in FIGS. 29(B) and 29(D), when the inquiry screen overlaps the avatar object 610, in order to erase the said inquiry screen for a user who wants to visually recognize the avatar object 610, an operation to "Purchase" or "Cancel" can be prompted. Note that the position where the inquiry screen is displayed may be set so as not to overlap a specific part (for example, the face, or the upper body, etc.) of the avatar object 610. For example, when displaying the inquiry screen, the position of a specific part of the avatar object 610 may be specified, a position that does not overlap with the said position may be specified, and the inquiry screen may be displayed. Thereby, it is possible to prevent, as much as possible, the visibility of the avatar object 610 from being reduced by the inquiry screen.

[0282] <Supplementary Note> Matters described in each of the above embodiments are appended below.

[0283] (Supplementary Note 1): According to an aspect of an embodiment shown in the present disclosure, a program executed in a viewing terminal (user terminal 100) including a processor, a memory, and a touch screen, the processor receiving display data for displaying an image in a virtual space in which an object including a character object operated by a performer is arranged (for example, steps S22 and S26 in FIG. 8); displaying an image in the virtual space on the touch screen based on the display data (for example, steps S25, S28, and S29 in FIG. 8, FIGS. 28 to 32); and executing control corresponding to the type of the object in response to a touch operation on the object arranged in the virtual space (FIG. 30).

[0284] (Appendix 2): (In Appendix 1), the executing step reflects an object of the same type as the object in the virtual space by receiving a touch operation on the object (step S96 in FIG. 30).

[0285] (Appendix 3): (In Appendix 2), the processor is caused to execute a step of displaying a first screen for asking whether to purchase the object according to the object targeted by the touch operation (steps S84 and S85 in FIG. 30).

[0286] (Appendix 4): (In Appendix 3), the charging amount of the object is displayed on the first screen (FIG. 29(B)).

[0287] (Appendix 5): (In any one of Appendix 1 to Appendix 4), the executing step switches an object associated with a touched portion of the character object to another object in response to a touch operation on the character object (step S97 in FIG. 30).

[0288] (Appendix 6): In (Appendix 5), cause the processor to execute the step of displaying a second screen that asks whether or not to purchase the other object according to the other object (Steps S89 and S85 in FIG. 30).

[0289] (Appendix 7): In (Appendix 6), the charging amount of the other object is displayed on the second screen (FIG. 29(D)).

[0290] (Appendix 8): In any one of (Appendix 1) to (Appendix 7), the step to be executed causes the server to transmit position information for the performer to identify the position of the touch operation according to the touch operation on the character object (Step S87 in FIG. 30).

[0291] (Appendix 9): According to an aspect of an embodiment shown in the present disclosure, a method executed by a terminal device (user terminal 100) including a processor, a memory, and a touch screen, the method including receiving display data for enabling display of an image in a virtual space in which an object including a character object operated by a performer is arranged (for example, Steps S22 and S26 in FIG. 8); displaying the image in the virtual space on the touch screen based on the display data (for example, Steps S25, S28, S29 in FIG. 8, FIGS. 28 to 32); and executing control according to the type of the object in response to a touch operation on the object arranged in the virtual space (FIG. 30).

[0292] (Appendix 10): According to an aspect of an embodiment shown in the present disclosure, there is provided a terminal device (user terminal 100) including a processor, a memory, and a touch screen, the terminal device being configured to receive display data for displaying an image in a virtual space in which an object including a character object operated by a performer is arranged (for example, steps S22 and S26 in FIG. 8), display the image in the virtual space on the touch screen based on the display data (for example, steps S25, S28, and S29 in FIG. 8, FIGS. 28 to 32), and execute control corresponding to the type of the object in response to a touch operation on the object arranged in the virtual space (FIG. 30).

[0293] 〔Example of Realization by Software〕 The control blocks of the user terminal 100, the server 200, the game play terminal 300 (HMD set 1000), and the distribution terminal 400 (particularly the control units 110, 210, 310, and 410) may be realized by a logic circuit (hardware) formed in an integrated circuit (IC chip) or the like, or may be realized by software.

[0294] In the latter case, the user terminal 100, the server 200, the game play terminal 300 (HMD set 1000), and the distribution terminal 400 are each provided with a computer that executes program instructions of software that realizes each function. This computer includes, for example, one or more processors and a computer-readable recording medium that stores the above program. Then, in the above computer, when the above processor reads and executes the above program from the above recording medium, the object of the present invention is achieved. As the above processor, for example, a CPU (Central Processing Unit) can be used. As the above recording medium, "non-temporary tangible media" such as ROM (Read Only Memory), tapes, disks, cards, semiconductor memories, programmable logic circuits, etc. can be used. Further, it may further include a RAM (Random Access Memory) for expanding the above program. Further, the above program may be supplied to the above computer via any transmission medium (communication network, broadcast wave, etc.) capable of transmitting the program. Note that one aspect of the present invention can also be realized in the form of a data signal embedded in a carrier wave, in which the above program is embodied by electronic transmission.

[0295] The present invention is not limited to the above-described embodiments, and various modifications are possible within the scope shown in the claims. Embodiments obtained by appropriately combining the technical means disclosed in different embodiments are also included in the technical scope of the present invention.

Explanation of Reference Numerals

[0296] 1 System, 2 Network, 3, 3A, 3B User (First User), 4 Player (Performer), 10, 20, 30, 40 Processor, 11, 21, 31, 41 Memory, 12, 22, 32, 42 Storage, 13, 23, 33, 43 Communication IF, 14, 24, 34, 44 Input / Output IF, 15, 45 Touch Screen, 17 Camera, 18 Distance Measuring Sensor, 51 Monitor, 52 Gaze Sensor, 53 First Camera, 54 Second Camera, 55 Microphone, 56 Speaker, 100, 100A, 100B, 100C User Terminal (Computer, First Computer, First Information Processing Device), 110, 210, 310, 410 Control Unit (First Control Unit, Second Control Unit), 111, 311, 413 Operation Reception Unit, 112, 312, 412 Display Control Unit, 113, 313 UI Control Unit, 114, 314 Animation Generation Unit, 115, 315 Game Progression Unit, 116, 316 Virtual Space Control Unit, 117 Video Playback Unit, 120, 220, 320, 420 Storage Unit (First Storage Unit, Second Storage Unit), 131, 231, 331 Game Program (Program, First Program), 132, 232, 332 Game Information, 133, 233, 333 User Information, 151, 451 Input Unit, 152, 452 Display Unit (Display), 200 Server, 211 Communication Mediation Unit, 212 Log Generation Unit, 213 List Generation Unit, 234, 421 User List, 300 Gameplay Terminal (External Device, Second External Device), 317 Reaction Processing Unit, 400 Distribution Terminal (External, First External Device, Computer, Second Information Processing Device), 411 Communication Control Unit, 414 Voice Reception Unit, 415 Motion Identification Unit, 416 Motion Instruction Data Generation Unit, 422 Motion List, 423 Distribution Program (Program, Second Program), 540, 1020, 1021 Controller, 500 HMD, 510 HMD Sensor, 520 Motion Sensor, 530 Display, 600A, 600B Virtual Space, 610 Avatar Object (Character), 620A, 620B Virtual Camera, 631, 632, 633, 634 Object, 640A, 640B Field of View Region, 650, 660 Field of View Image, 671 Enemy Object, 672, 673 Obstacle Object, 674 Effect Object, 691, 692 Utterance, 701, 702, 703A, 70B,704A, 704B, 705, 706, 711, 711A, 711B, 711C, 711D, 722, 723, 745, 745A, 745B, 745C, 752, 762, 763, 930, 2011, 2022, 2031, 2032, 2033, 2034, 2037, 2038, 2051, 2063, 2072, 2073, 2075 UI images (message UI, UI), 721 download screen, 731 user list screen (list), 732, 732A, 732B, 732C, 742, 742A, 742B, 742C record images, 733, 733A, 733B, 733C user names, 734, 734A, 734B, 734C tag information, 735, 735A, 735B, 735C icons, 741 motion list screen (options), 743, 743A, 743B, 743C motion names, 744, 744A, 744B, 744C, 753 motion images, 751 distribution screen, 761 distribution completion screen, 810A, 810B motion videos, 820A, 820B spoken voices, 910A, 910B videos, 920A, 920B voices, 1000 HMD set, 1010 object, 1030 memory medium,

Claims

1. A computer, a receiving means for receiving display data for enabling display of an image in a virtual space in which objects including a character object operated by a performer are arranged; functioning as a display control means for displaying an image in the virtual space on a screen based on the display data; the display control means is an object arranged in the virtual space, and in response to a selection operation on an object that can be input into the virtual space by a user, displays a screen for inquiring about the purchase of an object that can be input into the virtual space; the computer, a receiving means for receiving a process in which the user purchases a purchase target by an operation on a screen for inquiring about the purchase of an object that can be input into the virtual space; further functioning as, wherein the inputtable object is an object associated with the character object operated by the performer, program.

2. a receiving means for receiving display data for enabling display of an image in a virtual space in which objects including a character object operated by a performer are arranged; a display control means for displaying an image in the virtual space on a screen based on the display data; comprising, the display control means is an object arranged in the virtual space, and in response to a selection operation on an object that can be input into the virtual space by a user, displays a screen for inquiring about the purchase of an object that can be input into the virtual space; a receiving means for receiving a process in which the user purchases a purchase target by an operation on a screen for inquiring about the purchase of an object that can be input into the virtual space; further comprising, wherein the inputtable object is an object associated with the character object operated by the performer, information processing system.

Images