Distribution program and system

Abstract

To provide distribution program and system capable of further enhancing a viewer's liking for content.SOLUTION: A computer distributes data for enabling a game play video of a game cleared when a predetermined attainment condition is established to be displayed on a user terminal. At this time, the computer has a support effect for supporting establishment of a predetermined condition upon receipt of operation information transmitted from a viewing terminal on the basis of a predetermined operation by a viewer that views the game play video. The support effect includes the arrangement of a specific object in a predetermined virtual space, and adjusts a degree of difficulty to establish the predetermined condition by the arrangement of the specific object in the predetermined virtual space.SELECTED DRAWING: Figure 21

Description

[Technical Field]

[0001] The present invention relates to a distribution program and system. [Background technology]

[0002] Patent Document 1 describes an information processing system that distributes live video to a viewing terminal. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Laid-Open No. 2012-120098 Summary of the Invention [Problem to be solved by the invention]

[0004] However, the above-mentioned conventional techniques have room for further increasing viewers' preference for content.

[0005] The present invention has been devised in view of the above circumstances, and its object is to provide a distribution program and system that can further increase viewers' preference for content. [Means for solving the problem]

[0006] According to one aspect of an embodiment of the present disclosure, there is provided a distribution program executed by a computer that distributes data for enabling a viewing terminal to display gameplay footage of a game that is cleared by fulfilling a predetermined condition in a predetermined virtual space, the program causing the computer to function as a means for exerting a support action to assist in fulfilling the predetermined condition in response to receiving operation information transmitted from the viewing terminal based on a predetermined operation by a viewer watching the gameplay footage, the support action including placing a specific object in the predetermined virtual space, and the means for exerting the support action being capable of adjusting the difficulty of fulfilling the predetermined condition by placing the specific object in the predetermined virtual space.

[0007] In addition, multiple types of specific objects are provided, the operation information includes information for specifying the type of specific object selected by the viewer, and the means for exerting the support effect places, as the support effect, the specific object of the type specified from the operation information within a predetermined virtual space.

[0008] The data also includes at least motion data indicating the movements of the broadcaster, the game is a game in which a player object that operates based on the motion data fights other objects, the computer functions as a means for identifying the number of viewers of the viewing terminal displaying the gameplay video, and the means for providing support can adjust the difficulty level by changing at least one of the abilities of the player object, the abilities of the other objects, and the number of other objects that appear, depending on the number of viewers.

[0009] According to one aspect of one embodiment, a system includes a distribution device that distributes data to enable the display of gameplay footage of a game that is cleared by fulfilling specified conditions within a specified virtual space, and a viewing terminal that displays the gameplay footage based on the data distributed from the distribution device, wherein the viewing terminal transmits operation information based on specified operations by a viewer watching the gameplay footage, and the distribution device has means for exerting a support action to assist in fulfilling the specified conditions in response to receiving the operation information transmitted from the viewing terminal, the support action including placing a specified object within the specified virtual space, and the means for exerting the support action is capable of adjusting the difficulty of fulfilling the specified conditions by placing a specified object within the specified virtual space. [Effects of the Invention]

[0010] According to the present invention, it is possible to further increase viewers' preference for content. [Brief explanation of the drawings]

[0011] [Figure 1] FIG. 1 is a diagram illustrating an outline of the configuration of an HMD system according to an embodiment. [Figure 2] FIG. 1 is a block diagram illustrating an example of a hardware configuration of a computer according to an embodiment. [Figure 3] FIG. 1 is a diagram conceptually illustrating a uvw field of view coordinate system set in an HMD according to an embodiment. [Figure 4] FIG. 1 is a diagram conceptually illustrating one mode of representing a virtual space according to an embodiment. [Figure 5] 1 is a top view of a user's head wearing an HMD according to an embodiment. FIG. [Figure 6] 10 is a diagram showing a YZ cross section of a field of view in a virtual space as viewed from the X direction. [Figure 7] 10 is a diagram showing an XZ cross section of a field of view in a virtual space as viewed from the Y direction. [Figure 8]FIG. 1 is a diagram showing an example of a virtual space defined in an HMD set according to an embodiment, and objects arranged in the virtual space. [Figure 9] FIG. 1A is a diagram showing a schematic configuration of a controller according to an embodiment, and FIG. 1B is a diagram showing an example of yaw, roll, and pitch directions defined with respect to a user's right hand according to an embodiment. [Figure 10] FIG. 2 is a block diagram illustrating an example of a hardware configuration of a server according to an embodiment. [Figure 11] FIG. 1 is a block diagram illustrating a modular configuration of a computer according to an embodiment. [Figure 12] FIG. 2 is a block diagram showing a detailed configuration of modules in a computer according to an embodiment. [Figure 13] FIG. 2 is a diagram illustrating a hardware configuration of a user terminal according to an embodiment. [Figure 14] (A) is a diagram showing an example of a virtual space defined in a user terminal according to an embodiment and an object placed in the virtual space, and (B) is a diagram showing an example of a field of view image displayed on the user terminal. [Figure 15] FIG. 2 is a block diagram showing functional configurations of a user terminal, a server, and an HMD set included in an HMD system according to an embodiment. [Figure 16] FIG. 2 is a diagram illustrating a detailed configuration of a virtual space control unit included in a user terminal according to an embodiment. [Figure 17] FIG. 2 is a diagram showing a detailed configuration of a virtual space control unit included in an HMD set according to an embodiment. [Figure 18] (A) is a diagram showing another example of a field of view image displayed on a user terminal, (B) is a diagram showing another example of a field of view image displayed on a user terminal, (C) is a diagram showing yet another example of a field of view image displayed on a user terminal, and (D) is a diagram showing another example of a field of view image displayed on a user terminal. [Figure 19]10 is a timing chart showing part of the operation of the user terminal and the computer in response to an item insertion operation. [Figure 20] 10 is a flowchart showing a part of the flow of processing executed in the HMD system. [Figure 21] 10 is a flowchart showing another part of the flow of the processing executed in the HMD system. DETAILED DESCRIPTION OF THE INVENTION

[0012] Hereinafter, embodiments of this technical idea will be described in detail with reference to the drawings. In the following description, the same components are denoted by the same reference numerals. Their names and functions are also the same. Therefore, detailed descriptions thereof will not be repeated. In one or more embodiments shown in this disclosure, elements included in each embodiment can be combined with each other, and the combined result also forms part of the embodiment shown in this disclosure.

[0013] [HMD system configuration] The configuration of an HMD (Head-Mounted Device) system 100 will be described with reference to Fig. 1. Fig. 1 is a diagram illustrating an outline of the configuration of HMD system 100 according to the present embodiment. HMD system 100 is provided as a system for home use or a system for business use.

[0014] The HMD system 100 includes a server 600, an HMD set 110, an external device 700, a network 2, and multiple user terminals 800. The HMD set 110 is configured to be able to communicate with the server 600, the external device 700, and the user terminals 800 via the network 2. The HMD set 110 includes an HMD 120, a computer 200, an HMD sensor 410, a display 430, and a controller 300. The HMD 120 includes a monitor 130, a gaze sensor 140, a first camera 150, a second camera 160, a microphone 170, and a speaker 180. The controller 300 may include a motion sensor 420.

[0015] In one aspect, the computer 200 can be connected to the Internet or other network 2, and can communicate with a server 600 or other computers connected to the network 2. Examples of other computers include a user terminal 800, an external device 700, and computers of other HMD sets 110. In another aspect, the HMD 120 can include a sensor 190 instead of the HMD sensor 410.

[0016] The HMD 120 is worn on the head of the player 5 and can provide a virtual space to the player 5 during operation. More specifically, the HMD 120 displays an image for the right eye and an image for the left eye on the monitor 130. When each eye of the player 5 views the respective image, the player 5 can recognize the image as a three-dimensional image based on the parallax between the two eyes. The HMD 120 can include both a so-called head-mounted display equipped with a monitor and a head-mounted device to which a smartphone or other terminal equipped with a monitor can be attached. The HMD 120 provides a virtual space for the player 5 to play a game that can be cleared by, for example, achieving a predetermined achievement condition. The player 5 is also referred to as a distributor because he distributes data that enables the game play screen to be displayed on the user terminal 800.

[0017] The game genre is not limited to a specific genre, and the HMD system 100 can execute games of any genre, such as sports-themed games such as tennis, table tennis, dodgeball, baseball, soccer, and hockey, puzzle games, quiz games, role-playing games (RPGs), adventure games, shooting games, simulation games, training games, and action games.

[0018] Furthermore, the play style of the game is not limited to a specific play style. The HMD system 100 can provide a virtual space in which games of any play style can be played. For example, the game may be a single-play game played by a single player 5, a multi-play game played by multiple players 5, or, among multi-play games, a competitive game in which multiple players 5 compete against each other, or a cooperative game in which multiple players 5 cooperate with each other.

[0019] The monitor 130 is realized, for example, as a non-transparent display device. In one aspect, the monitor 130 is disposed on the main body of the HMD 120 so as to be positioned in front of both eyes of the player 5. Therefore, when the player 5 views the three-dimensional image displayed on the monitor 130, the player 5 can become immersed in the virtual space. In one aspect, for example, an avatar object that the player 5 can control and other objects are placed in the virtual space. In another aspect, the monitor 130 can be realized as a liquid crystal monitor or an organic EL (Electro Luminescence) monitor provided in a so-called smartphone or other information display terminal.

[0020] In another aspect, the monitor 130 may be realized as a transmissive display device. In this case, the HMD 120 may be an open type such as glasses, rather than a closed type that covers the eyes of the player 5 as shown in FIG. 1 . The transmissive monitor 130 may be temporarily configured as a non-transmissive display device by adjusting its transmittance. The monitor 130 may include a configuration that simultaneously displays a portion of an image that constitutes a virtual space and the real space. For example, the monitor 130 may display an image of the real space captured by a camera mounted on the HMD 120, or may make the real space visible by setting the transmittance of a portion of the monitor 130 high.

[0021] In one aspect, monitor 130 may include a sub-monitor for displaying an image for the right eye and a sub-monitor for displaying an image for the left eye. In another aspect, monitor 130 may be configured to display an image for the right eye and an image for the left eye as a single image. In this case, monitor 130 includes a high-speed shutter. The high-speed shutter operates to alternately display an image for the right eye and an image for the left eye so that the image is recognized by only one of the eyes.

[0022] In one aspect, the HMD 120 includes a plurality of light sources (not shown). Each light source is realized, for example, by an LED (Light Emitting Diode) that emits infrared rays. The HMD sensor 410 has a position tracking function for detecting the movement of the HMD 120. More specifically, the HMD sensor 410 reads a plurality of infrared rays emitted by the HMD 120 and detects the position and tilt of the HMD 120 in real space.

[0023] In another aspect, the HMD sensor 410 may be realized by a camera. In this case, the HMD sensor 410 can detect the position and tilt of the HMD 120 by performing image analysis processing using image information of the HMD 120 output from the camera.

[0024] In another aspect, the HMD 120 may include the sensor 190 as a position detector instead of or in addition to the HMD sensor 410. The HMD 120 may use the sensor 190 to detect the position and tilt of the HMD 120 itself. For example, if the sensor 190 is an angular velocity sensor, a geomagnetic sensor, or an acceleration sensor, the HMD 120 may use any of these sensors instead of the HMD sensor 410 to detect the position and tilt of the HMD 120 itself. As an example, if the sensor 190 is an angular velocity sensor, the angular velocity sensor detects the angular velocity of the HMD 120 around three axes in real space over time. The HMD 120 calculates changes over time in the angles of the HMD 120 around the three axes based on the angular velocities, and further calculates the tilt of the HMD 120 based on the changes over time in the angles.

[0025] The gaze sensor 140 detects the direction in which the right and left eyes of the player 5 are looking. In other words, the gaze sensor 140 detects the gaze of the player 5. Detection of the gaze direction is achieved, for example, by a known eye tracking function. The gaze sensor 140 is achieved by a sensor having the eye tracking function. In certain aspects, the gaze sensor 140 preferably includes a sensor for the right eye and a sensor for the left eye. The gaze sensor 140 may be, for example, a sensor that irradiates the right and left eyes of the player 5 with infrared light and detects the rotation angle of each eyeball by receiving light reflected from the cornea and iris of the irradiated light. The gaze sensor 140 can detect the gaze of the player 5 based on the detected rotation angles.

[0026] The first camera 150 captures the lower part of the face of the player 5. More specifically, the first camera 150 captures the nose, mouth, etc. of the player 5. The second camera 160 captures the eyes, eyebrows, etc. of the player 5. The housing of the HMD 120 on the player 5 side is defined as the inside of the HMD 120, and the housing of the HMD 120 on the opposite side from the player 5 is defined as the outside of the HMD 120. In one aspect, the first camera 150 may be disposed outside the HMD 120, and the second camera 160 may be disposed inside the HMD 120. Images generated by the first camera 150 and the second camera 160 may be input to the computer 200. In another aspect, the first camera 150 and the second camera 160 may be implemented as a single camera, and the face of the player 5 may be captured by this single camera.

[0027] The microphone 170 converts the speech of the player 5 into an audio signal (electrical signal) and outputs it to the computer 200. The speaker 180 converts the audio signal into sound and outputs it to the player 5. In another aspect, the HMD 120 may include earphones instead of the speaker 180.

[0028] The controller 300 is connected to the computer 200 by wire or wirelessly. The controller 300 accepts input of commands from the player 5 to the computer 200. In one aspect, the controller 300 is configured to be able to be held by the player 5. In another aspect, the controller 300 is configured to be able to be attached to the body or clothing of the player 5. In yet another aspect, the controller 300 may be configured to output at least one of vibration, sound, and light based on a signal transmitted from the computer 200. In yet another aspect, the controller 300 accepts operations from the player 5 to control the position and movement of an object placed in a virtual space.

[0029] In one aspect, the controller 300 includes multiple light sources. Each light source is realized, for example, by an LED that emits infrared light. The HMD sensor 410 has a position tracking function. In this case, the HMD sensor 410 reads multiple infrared rays emitted by the controller 300 and detects the position and tilt of the controller 300 in real space, i.e., the player's body movements. In another aspect, the HMD sensor 410 may be realized by a camera. In this case, the HMD sensor 410 can detect the position and tilt of the controller 300 by performing image analysis processing using image information of the controller 300 output from the camera.

[0030] In one aspect, the motion sensor 420 is attached to the hand of the player 5 and detects the movement of the hand of the player 5. For example, the motion sensor 420 detects the rotation speed, number of rotations, etc. of the hand. The detected signal is sent to the computer 200. The motion sensor 420 is provided, for example, in the controller 300. In one aspect, the motion sensor 420 is provided, for example, in the controller 300 configured to be held by the player 5. In another aspect, for safety in real space, the controller 300 is attached to something that is worn on the hand of the player 5, such as a glove, so that it will not easily fly away. In yet another aspect, a sensor not attached to the player 5 may detect the movement of the hand of the player 5. For example, a signal from a camera capturing an image of the player 5 may be input to the computer 200 as a signal representing the movement of the player 5. The motion sensor 420 and the computer 200 are connected to each other wirelessly, for example. In the case of wireless communication, the communication format is not particularly limited, and for example, Bluetooth (registered trademark) or other known communication methods may be used.

[0031] The display 430 displays an image similar to the image displayed on the monitor 130. This allows a person involved in the data distribution other than the player 5 wearing the HMD 120 (also referred to as a distributor, similar to the player 5) to view the same image as the player 5. The image displayed on the display 430 does not need to be a three-dimensional image, and may be an image for the right eye or an image for the left eye. Examples of the display 430 include a liquid crystal display and an organic EL monitor.

[0032] Server 600 may transmit a program to computer 200. In another aspect, server 600 may distribute data for enabling the display of a virtual space provided by computer 200 to a plurality of user terminals (e.g., user terminals 800A, 800C, and 800D) connectable to server 600. Note that computer 200 may distribute the data without going through server 600.

[0033] The server 600 may communicate with other computers 200 to provide virtual reality to the HMDs 120 used by other players. For example, in an amusement facility, when multiple players play a multiplayer game, each computer 200 communicates signals based on the actions of each player with the other computers 200 via the server 600, allowing multiple players to enjoy a common game in the same virtual space. Each computer 200 may also communicate signals based on the actions of each player with the other computers 200 without going through the server 600.

[0034] The external device 700 may be any device that can communicate with the computer 200. The external device 700 may be, for example, a device that can communicate with the computer 200 via the network 2, or a device that can directly communicate with the computer 200 via short-range wireless communication or a wired connection. Examples of the external device 700 include, but are not limited to, smart devices, PCs (Personal Computers), and peripheral devices of the computer 200.

[0035] [Computer hardware configuration] A computer 200 according to this embodiment will be described with reference to Fig. 2. Fig. 2 is a block diagram showing an example of a hardware configuration of computer 200 according to this embodiment. Computer 200 includes, as main components, a processor 210, a memory 220, a storage 230, an input / output interface (hereinafter, IF) 240, and a communication IF 250. Each component is connected to a bus 260.

[0036] Processor 210 executes a series of instructions included in a program stored in memory 220 or storage 230 based on a signal provided to computer 200 or based on the establishment of a predetermined condition. In one aspect, processor 210 is realized as a central processing unit (CPU), a graphics processing unit (GPU), a microprocessor unit (MPU), a field-programmable gate array (FPGA), or other device.

[0037] Memory 220 temporarily stores programs and data. Programs are loaded from storage 230, for example. Data includes data input to computer 200 and data generated by processor 210. In one aspect, memory 220 is realized as RAM (Random Access Memory) or other volatile memory.

[0038] The storage 230 permanently stores programs and data. The storage 230 is realized, for example, as a ROM (Read-Only Memory), a hard disk drive, a flash memory, or other non-volatile storage device. The programs stored in the storage 230 include a program for providing a virtual space in the HMD system 100, a simulation program, a game program, a user authentication program, and a program for realizing communication with other computers 200. The data stored in the storage 230 includes data and objects for defining the virtual space.

[0039] In another aspect, storage 230 may be realized as a removable storage device such as a memory card. In yet another aspect, a configuration may be used in which programs and data stored in an external storage device are used instead of storage 230 built into computer 200. With such a configuration, for example, in a situation where multiple HMD systems 100 are used, such as an amusement facility, it becomes possible to collectively update programs and data.

[0040] The input / output IF 240 communicates signals between the HMD 120, the HMD sensor 410, the motion sensor 420, and the display 430. The monitor 130, the gaze sensor 140, the first camera 150, the second camera 160, the microphone 170, and the speaker 180 included in the HMD 120 can communicate with the computer 200 via the input / output IF 240 of the HMD 120. In one aspect, the input / output IF 240 is realized using a terminal such as a Universal Serial Bus (USB), a Digital Visual Interface (DVI), a High-Definition Multimedia Interface (HDMI (registered trademark)), or the like. The input / output IF 240 is not limited to those described above.

[0041] In one aspect, the input / output IF 240 may further communicate with the controller 300. For example, the input / output IF 240 receives input of signals output from the controller 300 and the motion sensor 420. In another aspect, the input / output IF 240 sends instructions output from the processor 210 to the controller 300. The instructions instruct the controller 300 to vibrate, output sound, emit light, etc. Upon receiving the instructions, the controller 300 executes one of the following in response to the instructions: vibration, sound output, or light emission.

[0042] The communication IF 250 is connected to the network 2 and communicates with other computers (e.g., the server 600) connected to the network 2. In one aspect, the communication IF 250 is realized as, for example, a wired communication IF such as a local area network (LAN), or a wireless communication IF such as WiFi (Wireless Fidelity), Bluetooth (registered trademark), or NFC (Near Field Communication). The communication IF 250 is not limited to those described above.

[0043] In one aspect, the processor 210 accesses the storage 230, loads one or more programs stored in the storage 230 into the memory 220, and executes a series of instructions included in the programs. The one or more programs may include an operating system for the computer 200, an application program for providing a virtual space, game software executable in the virtual space, etc. The processor 210 sends a signal for providing the virtual space to the HMD 120 via the input / output IF 240. The HMD 120 displays an image on the monitor 130 based on the signal.

[0044] 2 shows a configuration in which the computer 200 is provided outside the HMD 120, but in another aspect, the computer 200 may be built into the HMD 120. As an example, a portable information communication terminal (e.g., a smartphone) including the monitor 130 may function as the computer 200.

[0045] The computer 200 may be configured to be shared by multiple HMDs 120. With such a configuration, for example, the same virtual space can be provided to multiple players, allowing each player to enjoy the same application as other players in the same virtual space.

[0046] In one embodiment, a real coordinate system, which is a coordinate system in real space, is set in advance in the HMD system 100. The real coordinate system has three reference directions (axes) parallel to the vertical direction in real space, the horizontal direction perpendicular to the vertical direction, and the front-to-back direction perpendicular to both the vertical and horizontal directions. The horizontal direction, vertical direction (up-down direction), and front-to-back direction in the real coordinate system are defined as the x-axis, y-axis, and z-axis, respectively. More specifically, in the real coordinate system, the x-axis is parallel to the horizontal direction in real space. The y-axis is parallel to the vertical direction in real space. The z-axis is parallel to the front-to-back direction in real space.

[0047] In one aspect, the HMD sensor 410 includes an infrared sensor. When the infrared sensor detects infrared rays emitted from each light source of the HMD 120, it detects the presence of the HMD 120. The HMD sensor 410 further detects the position and tilt (orientation) of the HMD 120 in real space in accordance with the movement of the player 5 wearing the HMD 120, based on the values ​​of each point (each coordinate value in the real coordinate system). More specifically, the HMD sensor 410 can detect changes over time in the position and tilt of the HMD 120 using each value detected over time.

[0048] Each tilt of the HMD 120 detected by the HMD sensor 410 corresponds to each tilt around the three axes of the HMD 120 in the real coordinate system. The HMD sensor 410 sets a uvw field of view coordinate system for the HMD 120 based on the tilt of the HMD 120 in the real coordinate system. The uvw field of view coordinate system set for the HMD 120 corresponds to a viewpoint coordinate system when the player 5 wearing the HMD 120 views an object in the virtual space.

[0049] [uvw field of view coordinate system] The uvw field of view coordinate system will be described with reference to Fig. 3. Fig. 3 is a conceptual diagram showing the uvw field of view coordinate system set in the HMD 120 according to an embodiment. The HMD sensor 410 detects the position and tilt of the HMD 120 in the real coordinate system when the HMD 120 is started up. The processor 210 sets the uvw field of view coordinate system in the HMD 120 based on the detected values.

[0050] 3, the HMD 120 sets a three-dimensional uvw field of view coordinate system with the head of the player 5 wearing the HMD 120 as its center (origin). More specifically, the HMD 120 tilts the horizontal, vertical, and front-to-back directions (x-axis, y-axis, z-axis) that define the real coordinate system around each axis by the tilt of the HMD 120 around each axis in the real coordinate system, and sets these three directions as the pitch axis (u-axis), yaw axis (v-axis), and roll axis (w-axis) of the uvw field of view coordinate system in the HMD 120.

[0051] In a certain situation, when the player 5 wearing the HMD 120 stands upright and looks straight ahead, the processor 210 sets a uvw field of view coordinate system parallel to the real coordinate system in the HMD 120. In this case, the horizontal direction (x-axis), vertical direction (y-axis), and front-to-back direction (z-axis) in the real coordinate system coincide with the pitch axis (u-axis), yaw axis (v-axis), and roll axis (w-axis) of the uvw field of view coordinate system in the HMD 120.

[0052] After the uvw field of view coordinate system is set in the HMD 120, the HMD sensor 410 can detect the tilt of the HMD 120 in the set uvw field of view coordinate system based on the movement of the HMD 120. In this case, the HMD sensor 410 detects the pitch angle (θu), yaw angle (θv), and roll angle (θw) of the HMD 120 in the uvw field of view coordinate system as the tilt of the HMD 120. The pitch angle (θu) represents the tilt angle of the HMD 120 around the pitch axis in the uvw field of view coordinate system. The yaw angle (θv) represents the tilt angle of the HMD 120 around the yaw axis in the uvw field of view coordinate system. The roll angle (θw) represents the tilt angle of the HMD 120 around the roll axis in the uvw field of view coordinate system.

[0053] The HMD sensor 410 sets, in the HMD 120, a uvw field of view coordinate system for the HMD 120 after the HMD 120 has moved, based on the detected tilt of the HMD 120. The relationship between the HMD 120 and the uvw field of view coordinate system of the HMD 120 is always constant, regardless of the position and tilt of the HMD 120. When the position and tilt of the HMD 120 change, the position and tilt of the uvw field of view coordinate system of the HMD 120 in the real coordinate system change in conjunction with the change in the position and tilt.

[0054] In one aspect, the HMD sensor 410 may identify the position of the HMD 120 in real space as a relative position with respect to the HMD sensor 410 based on the light intensity of the infrared light acquired based on the output from the infrared sensor and the relative positional relationship between multiple points (e.g., the distance between each point, etc.). The processor 210 may determine the origin of the uvw field of view coordinate system of the HMD 120 in real space (actual coordinate system) based on the identified relative position.

[0055] [Virtual Space] The virtual space will be further described with reference to FIG. 4. FIG. 4 is a conceptual diagram illustrating one manner of expressing virtual space 11 according to an embodiment. Virtual space 11 has a spherical structure that covers the entire 360-degree area from center 12. To avoid complicating the explanation, FIG. 4 illustrates only the upper half of the celestial sphere in virtual space 11. Meshes are defined in virtual space 11. The position of each mesh is defined in advance as a coordinate value in an XYZ coordinate system, which is a global coordinate system defined in virtual space 11. Computer 200 associates each partial image that constitutes panoramic image 13 (still image, video, etc.) that can be deployed in virtual space 11 with a corresponding mesh in virtual space 11.

[0056] In a certain aspect, an XYZ coordinate system is defined in virtual space 11 with center 12 as the origin. The XYZ coordinate system is, for example, parallel to the real coordinate system. The horizontal direction, vertical direction (up-down direction), and front-to-back direction in the XYZ coordinate system are defined as the X-axis, Y-axis, and Z-axis, respectively. Therefore, the X-axis (horizontal direction) of the XYZ coordinate system is parallel to the x-axis of the real coordinate system, the Y-axis (vertical direction) of the XYZ coordinate system is parallel to the y-axis of the real coordinate system, and the Z-axis (front-to-back direction) of the XYZ coordinate system is parallel to the z-axis of the real coordinate system.

[0057] When the HMD 120 is started up, i.e., in the initial state of the HMD 120, the virtual camera 14 is placed at the center 12 of the virtual space 11. In a certain stage, the processor 210 displays an image captured by the virtual camera 14 on the monitor 130 of the HMD 120. The virtual camera 14 moves in the virtual space 11 in conjunction with the movement of the HMD 120 in the real space. This allows changes in the position and tilt of the HMD 120 in the real space to be reproduced in the virtual space 11 in the same manner.

[0058] A uvw field of view coordinate system is defined for the virtual camera 14, as in the case of the HMD 120. The uvw field of view coordinate system of the virtual camera 14 in the virtual space 11 is defined so as to be linked to the uvw field of view coordinate system of the HMD 120 in the real space (actual coordinate system). Therefore, when the tilt of the HMD 120 changes, the tilt of the virtual camera 14 also changes accordingly. The virtual camera 14 can also move in the virtual space 11 in conjunction with the movement in the real space of the player 5 wearing the HMD 120.

[0059] Processor 210 of computer 200 defines field of view 15 in virtual space 11 based on the position and tilt (reference line of sight 16) of virtual camera 14. Field of view 15 corresponds to the area of ​​virtual space 11 that is visible to player 5 wearing HMD 120. In other words, the position of virtual camera 14 can be said to be the viewpoint of player 5 in virtual space 11.

[0060] The line of sight of the player 5 detected by the gaze sensor 140 is the direction in the viewpoint coordinate system when the player 5 views an object. The uvw field of view coordinate system of the HMD 120 is equal to the viewpoint coordinate system when the player 5 views the monitor 130. The uvw field of view coordinate system of the virtual camera 14 is linked to the uvw field of view coordinate system of the HMD 120. Therefore, in a certain situation, the HMD system 100 can regard the line of sight of the player 5 detected by the gaze sensor 140 as the line of sight of the player 5 in the uvw field of view coordinate system of the virtual camera 14.

[0061] [Player's gaze] Determining the line of sight of the player 5 will be described with reference to Fig. 5. Fig. 5 is a diagram showing the head of the player 5 wearing the HMD 120 according to an embodiment from above.

[0062] In one situation, the gaze sensor 140 detects the gaze of each of the right and left eyes of the player 5. In one situation, when the player 5 is looking at something close, the gaze sensor 140 detects gazes R1 and L1. In another situation, when the player 5 is looking at something far away, the gaze sensor 140 detects gazes R2 and L2. In this case, the angle formed by the gazes R2 and L2 with respect to the roll axis w is smaller than the angle formed by the gazes R1 and L1 with respect to the roll axis w. The gaze sensor 140 transmits the detection results to the computer 200.

[0063] When the computer 200 receives the detection values ​​of the lines of sight R1 and L1 from the gaze sensor 140 as the gaze detection result, it identifies the gaze point N1, which is the intersection of the lines of sight R1 and L1, based on the detection values. On the other hand, when the computer 200 receives the detection values ​​of the lines of sight R2 and L2 from the gaze sensor 140, it identifies the intersection of the lines of sight R2 and L2 as the gaze point. The computer 200 identifies the line of sight N0 of the player 5 based on the position of the identified gaze point N1. For example, the computer 200 detects the line of sight N0 as the direction of a line passing through the midpoint of a line connecting the right eye R and left eye L of the player 5 and the gaze point N1. The line of sight N0 is the direction in which the player 5 is actually looking with both eyes. The line of sight N0 corresponds to the direction in which the player 5 is actually looking with respect to the field of view 15.

[0064] In another aspect, the HMD system 100 may include a television broadcast receiving tuner. With this configuration, the HMD system 100 can display television programs in the virtual space 11.

[0065] In yet another aspect, the HMD system 100 may be provided with a communication circuit for connecting to the Internet or a telephone function for connecting to a telephone line.

[0066] [Visibility area] The field of view 15 will be described with reference to Figures 6 and 7. Figure 6 is a diagram illustrating a YZ cross section of the field of view 15 in virtual space 11 as viewed from the X direction. Figure 7 is a diagram illustrating an XZ cross section of the field of view 15 in virtual space 11 as viewed from the Y direction.

[0067] 6, the field of view 15 in the YZ cross section includes an area 18. The area 18 is defined by the position of the virtual camera 14, the reference line of sight 16, and the YZ cross section of the virtual space 11. The processor 210 defines the range including the polar angle α centered on the reference line of sight 16 in the virtual space as the area 18.

[0068] 7, the field of view 15 in the XZ cross section includes an area 19. The area 19 is defined by the position of the virtual camera 14, the reference line of sight 16, and the XZ cross section of the virtual space 11. The processor 210 defines a range including an azimuth angle β centered on the reference line of sight 16 in the virtual space 11 as the area 19. The polar angles α and β are determined according to the position of the virtual camera 14 and the tilt (orientation) of the virtual camera 14.

[0069] In a certain situation, the HMD system 100 provides the player 5 with a field of view in the virtual space 11 by displaying a field of view image 17 on the monitor 130 based on a signal from the computer 200. The field of view image 17 is an image corresponding to a portion of the panoramic image 13 that corresponds to the field of view area 15. When the player 5 moves the HMD 120 worn on his / her head, the virtual camera 14 also moves in conjunction with the movement. As a result, the position of the field of view 15 in the virtual space 11 changes. As a result, the field of view image 17 displayed on the monitor 130 is updated to an image of the panoramic image 13 that is superimposed on the field of view 15 in the direction in which the player 5 is facing in the virtual space 11. The player 5 can view a desired direction in the virtual space 11.

[0070] In this way, the tilt of virtual camera 14 corresponds to the line of sight (reference line of sight 16) of player 5 in virtual space 11, and the position at which virtual camera 14 is placed corresponds to the viewpoint of player 5 in virtual space 11. Therefore, by changing the position or tilt of virtual camera 14, the image displayed on monitor 130 is updated and the field of view of player 5 is moved.

[0071] While wearing the HMD 120, the player 5 can view only the panoramic image 13 unfolded in the virtual space 11, without viewing the real world. Therefore, the HMD system 100 can give the player 5 a highly immersive feeling in the virtual space 11.

[0072] In a certain situation, the processor 210 may move the virtual camera 14 in the virtual space 11 in conjunction with the movement in real space of the player 5 wearing the HMD 120. In this case, the processor 210 specifies the image area (field of view 15) to be projected onto the monitor 130 of the HMD 120 based on the position and tilt of the virtual camera 14 in the virtual space 11.

[0073] In one aspect, virtual camera 14 may include two virtual cameras, i.e., a virtual camera for providing an image for the right eye and a virtual camera for providing an image for the left eye. An appropriate parallax is set for the two virtual cameras so that player 5 can recognize three-dimensional virtual space 11. In another aspect, virtual camera 14 may be realized by a single virtual camera. In this case, an image for the right eye and an image for the left eye may be generated from an image obtained by the single virtual camera. In this embodiment, the technical idea of ​​the present disclosure is illustrated assuming that virtual camera 14 includes two virtual cameras and is configured so that a roll axis (w) generated by combining the roll axes of the two virtual cameras is adapted to the roll axis (w) of HMD 120.

[0074] [Objects placed in virtual space] An example of virtual space 11 will be described with reference to FIG. 8. Multiple types of objects are arranged in virtual space 11. The multiple types of objects include an avatar object 6 representing the player 5, controllable objects such as a virtual hand operated by controller 300, character objects, and background objects such as forests, mountains, and other landscapes, cityscapes, and animals that are arranged according to the progress of the game story. FIG. 8 shows an example in which avatar object 6 and objects A to D are arranged. Examples of objects A to D include, but are not limited to, character objects and background objects. Avatar object 6 moves and is arranged at a position corresponding to the movement of player 5. Objects A to D (hereinafter also referred to as other objects) move and are arranged at positions corresponding to the progress of the game based on the game program.

[0075] For example, in a fighting game in which the avatar object 6 fights against an enemy character, the other objects would be the enemy character's object, obstacle objects that hinder the enemy character's attacks, etc. In a puzzle game in which the avatar object rearranges the arrangement of blocks, the other objects would be a plurality of block objects with different colors and shapes.

[0076] Furthermore, in an escape game in which an avatar object escapes from a locked room, other objects correspond to furniture objects (tables, chairs, dressers, beds, etc.) placed in the room, objects of items needed for escape (keys, diaries, etc.), etc. In a music game in which an avatar object plays a musical instrument, other objects correspond to musical instrument objects (piano, violin, etc.), objects of other characters performing together with the avatar object, etc.

[0077] [controller] An example of the controller 300 will be described with reference to Fig. 9. Fig. 9 is a diagram illustrating a schematic configuration of the controller 300 according to an embodiment.

[0078] As shown in FIG. 9, in one aspect, the controller 300 may include a right controller 300R and a left controller (not shown). The right controller 300R is operated with the right hand of the player 5. The left controller is operated with the left hand of the player 5. In one aspect, the right controller 300R and the left controller are configured symmetrically as separate devices. Therefore, the player 5 can freely move both the right hand holding the right controller 300R and the left hand holding the left controller. In another aspect, the controller 300 may be an integrated controller that can be operated with both hands. The right controller 300R will be described below.

[0079] The right controller 300R includes a grip 310, a frame 320, and a top panel 330. The grip 310 is configured to be held by the right hand of the player 5. For example, the grip 310 can be held by the palm and three fingers (middle finger, ring finger, and little finger) of the right hand of the player 5.

[0080] The grip 310 includes buttons 340 and 350 and a motion sensor 420. The button 340 is located on the side of the grip 310 and is operated by the middle finger of the right hand. The button 350 is located on the front of the grip 310 and is operated by the index finger of the right hand. In some aspects, the buttons 340 and 350 are configured as trigger-type buttons. The motion sensor 420 is built into the housing of the grip 310. If the movements of the player 5 can be detected from around the player 5 by a camera or other device, the grip 310 does not need to include the motion sensor 420.

[0081] The frame 320 includes multiple infrared LEDs 360 arranged along its circumference. The infrared LEDs 360 emit infrared light in accordance with the progress of a program that uses the controller 300 while the program is being executed. The infrared light emitted from the infrared LEDs 360 can be used to detect the positions and attitudes (tilt, direction) of the right controller 300R and the left controller. In the example shown in FIG. 9, the infrared LEDs 360 are arranged in two rows, but the number of rows is not limited to that shown in FIG. 9. An arrangement in one row or three or more rows may also be used.

[0082] The top surface 330 is equipped with buttons 370, 380 and an analog stick 390. The buttons 370, 380 are configured as push buttons. The buttons 370, 380 are operated by the thumb of the player's 5 right hand. In a certain situation, the analog stick 390 is operated in any direction within 360 degrees from the initial position (neutral position). Such operations include, for example, operations for moving an object placed in the virtual space 11.

[0083] In one aspect, the right controller 300R and the left controller include batteries for driving the infrared LED 360 and other components. Batteries include, but are not limited to, rechargeable, button-type, and dry cell batteries. In another aspect, the right controller 300R and the left controller can be connected to, for example, the USBIF of the computer 200. In this case, the right controller 300R and the left controller do not require batteries.

[0084] 9, for example, the yaw, roll, and pitch directions are defined for the right hand of player 5. When player 5 extends his thumb and index finger, the extending direction of the thumb is defined as the yaw direction, the extending direction of the index finger is defined as the roll direction, and the direction perpendicular to the plane defined by the axis of the yaw direction and the axis of the roll direction is defined as the pitch direction.

[0085] [Server hardware configuration] Server 600 according to this embodiment will be described with reference to Fig. 10. Fig. 10 is a block diagram showing an example of a hardware configuration of server 600 according to an embodiment. Server 600 includes, as main components, a processor 610, a memory 620, a storage 630, an input / output IF 640, and a communication IF 650. Each component is connected to a bus 660.

[0086] The processor 610 executes a series of instructions included in a program stored in the memory 620 or the storage 630 based on a signal provided to the server 600 or based on the establishment of a predetermined condition. In one aspect, the processor 610 is implemented as a CPU, a GPU, an MPU, an FPGA, or other device.

[0087] The memory 620 temporarily stores programs and data. The programs are loaded from, for example, the storage 630. The data includes data input to the server 600 and data generated by the processor 610. In one aspect, the memory 620 is implemented as a RAM or other volatile memory.

[0088] Storage 630 permanently stores programs and data. Storage 630 is realized, for example, as a ROM, a hard disk drive, a flash memory, or other non-volatile storage device. Programs stored in storage 630 may include programs for providing virtual spaces in each of HMD system 100 and user terminal 800, simulation programs, game programs, user authentication programs, and programs for realizing communication with computer 200 and user terminal 800. Data stored in storage 630 may include data and objects for defining the virtual space.

[0089] In another aspect, storage 630 may be realized as a removable storage device such as a memory card. In yet another aspect, a configuration may be used in which programs and data stored in an external storage device are used instead of storage 630 built into server 600. With such a configuration, for example, in a situation where multiple HMD systems 100 are used, such as an amusement facility, it becomes possible to collectively update programs and data.

[0090] The input / output IF 640 communicates signals with input / output devices. In one aspect, the input / output IF 640 is implemented using a terminal such as a USB, DVI, HDMI (registered trademark), or the like. The input / output IF 640 is not limited to the above.

[0091] The communication IF 650 is connected to the network 2 and communicates with each of the computer 200 and the user terminal 800 connected to the network 2. In one aspect, the communication IF 650 is realized as, for example, a wired communication IF such as a LAN, or a wireless communication IF such as Wi-Fi, Bluetooth (registered trademark), or NFC. The communication IF 650 is not limited to the above.

[0092] In one aspect, processor 610 accesses storage 630, loads one or more programs stored in storage 630 into memory 620, and executes a series of instructions included in the programs. The one or more programs may include an operating system for server 600, an application program for providing a virtual space, game software executable in the virtual space, etc. Processor 610 may send signals for providing the virtual space to each of computer 200 and user terminal 800 via input / output IF 640.

[0093] [HMD control device] The control device of the HMD 120 will be described with reference to Fig. 11. In one embodiment, the control device is realized by a computer 200 having a known configuration. Fig. 11 is a block diagram showing the modular configuration of the computer 200 according to one embodiment.

[0094] 11 , the computer 200 includes a control module 510, a rendering module 520, a memory module 530, and a communication control module 540. In one aspect, the control module 510 and the rendering module 520 are implemented by the processor 210. In another aspect, multiple processors 210 may operate as the control module 510 and the rendering module 520. The memory module 530 is implemented by the memory 220 or the storage 230. The communication control module 540 is implemented by the communication IF 250.

[0095] The control module 510 controls the virtual space 11 provided to the player 5. The control module 510 defines the virtual space 11 in the HMD system 100 using virtual space data representing the virtual space 11. The virtual space data is stored in, for example, the memory module 530. The control module 510 may generate the virtual space data or obtain the virtual space data from the server 600 or the like.

[0096] The control module 510 places an object in the virtual space 11 using object data representing the object (placement data (display data) for placing (displaying) the object). The object data is stored in, for example, the memory module 530. The control module 510 may generate the object data or obtain the object data from the server 600 or the like.

[0097] Based on an image including the player 5, the control module 510 places an avatar object resembling the player 5 in the virtual space 11. In one aspect, the control module 510 accepts a selection by the player 5 from among a plurality of types of avatar objects (e.g., objects resembling animals or deformed human objects) and places the avatar object in the virtual space 11. In one aspect, the control module 510 places an avatar object of the player 5 of another computer 200 connected via the network 2 in the virtual space 11. In another aspect, the control module 510 may place an avatar object of the player 5 in the virtual space 11.

[0098] The control module 510 determines the tilt of the HMD 120 based on the output of the HMD sensor 410. In another aspect, the control module 510 determines the tilt of the HMD 120 based on the output of the sensor 190 functioning as a motion sensor. The control module 510 detects organs (e.g., mouth, eyes, eyebrows) that make up the face of the player 5 from the images of the face of the player 5 generated by the first camera 150 and the second camera 160. The control module 510 detects the movement (shape) of each detected organ.

[0099] The control module 510 detects the line of sight of the player 5 in the virtual space 11 based on a signal from the gaze sensor 140. The control module 510 detects the viewpoint position (coordinate values ​​in the XYZ coordinate system) where the detected line of sight of the player 5 intersects with the celestial sphere of the virtual space 11. More specifically, the control module 510 detects the viewpoint position based on the line of sight of the player 5 defined in the uvw coordinate system and the position and inclination of the virtual camera 14. The control module 510 transmits the detected viewpoint position to the server 600. In another aspect, the control module 510 may be configured to transmit line of sight information representing the line of sight of the player 5 to the server 600. In such a case, the server 600 may calculate the viewpoint position based on the line of sight information received.

[0100] The control module 510 reflects the body movement of the player 5 detected by the HMD sensor 410 in the avatar object. For example, the control module 510 detects that the player 5 has crouched and causes the avatar object to crouch. The control module 510 reflects the detected movement of the facial organs in the face of the avatar object placed in the virtual space 11. In one aspect, the control module 510 reflects the movement of the controller 300 in the avatar object or the control object. In this case, the controller 300 includes a motion sensor, an acceleration sensor, or a plurality of light-emitting elements (e.g., infrared LEDs) for detecting the movement of the controller 300. In another aspect, the control module 510 may receive gaze information of another player 5 from the server 600 and reflect the gaze information in the gaze of the avatar object of the other player 5.

[0101] The control module 510 places objects in the virtual space 11, including a control object for receiving operations from the player 5 in the virtual space 11. The player 5 operates the control object to, for example, operate an object placed in the virtual space 11. In one aspect, the control object may include, for example, a hand object that is a virtual hand corresponding to the hand of the player 5. In one aspect, the control module 510 moves the hand object in the virtual space 11 in conjunction with the movement of the hand of the player 5 in real space based on the output of the motion sensor 420. In one aspect, the control object may correspond to the hand portion of an avatar object.

[0102] The control module 510 detects a collision when each of the objects placed in the virtual space 11 collides with another object. The control module 510 can, for example, detect the timing when the collision area of ​​one object comes into contact with the collision area of ​​another object, and performs a predetermined process when this detection is made. The control module 510 can detect the timing when two objects are no longer in contact with each other, and performs a predetermined process when this detection is made. The control module 510 can detect when two objects are in contact with each other. For example, when a control object comes into contact with another object, the control module 510 detects this contact and performs a predetermined process.

[0103] In one aspect, the control module 510 controls image display on the monitor 130 of the HMD 120. For example, the control module 510 places a virtual camera 14 in the virtual space 11. The control module 510 controls the position of the virtual camera 14 in the virtual space 11 and the tilt (direction) of the virtual camera 14. The control module 510 defines a field of view 15 according to the tilt of the head of the player 5 wearing the HMD 120 and the position of the virtual camera 14. The rendering module 520 generates a field of view image 17 to be displayed on the monitor 130 based on the determined field of view 15. The field of view image 17 generated by the rendering module 520 is output to the HMD 120 by the communication control module 540.

[0104] When the control module 510 detects an utterance made by the player 5 using the microphone 170 from the HMD 120, it identifies the computer 200 to which audio data corresponding to the utterance is to be sent. The audio data is sent to the computer 200 identified by the control module 510. When the control module 510 receives audio data from another user's computer 200 via the network 2, it outputs audio (utterance) corresponding to the audio data from the speaker 180.

[0105] The memory module 530 stores data used by the computer 200 to provide the virtual space 11 to the player 5. In one aspect, the memory module 530 stores space information, object information, and player information.

[0106] The space information includes one or more templates defined to provide the virtual space 11 .

[0107] The object information includes a plurality of panoramic images 13 that constitute the virtual space 11, and object data for placing objects in the virtual space 11. The panoramic images 13 may include still images and moving images. The panoramic images 13 may include images of unreal spaces and images of real spaces. Examples of images of unreal spaces include images generated by computer graphics.

[0108] The player information holds a player ID that identifies the player 5. The player ID may be, for example, an IP (Internet Protocol) address or a MAC (Media Access Control) address set in the computer 200 used by the player. In another aspect, the player ID may be set by the player. The player information includes a program for causing the computer 200 to function as a control device for the HMD system 100.

[0109] The data and programs stored in the memory module 530 are input by the player 5 of the HMD 120. Alternatively, the processor 210 downloads the programs or data from a computer (e.g., the server 600) operated by the business that provides the content, and stores the downloaded programs or data in the memory module 530.

[0110] The communication control module 540 can communicate with the server 600 and other information communication devices via the network 2.

[0111] In one aspect, the control module 510 and the rendering module 520 may be realized using, for example, Unity (registered trademark) provided by Unity Technologies, Inc. In another aspect, the control module 510 and the rendering module 520 may be realized as a combination of circuit elements that realize each process.

[0112] Processing in computer 200 is realized by hardware and software executed by processor 210. Such software may be pre-stored on a hard disk or other memory module 530. Software may be stored on a CD-ROM or other computer-readable nonvolatile data recording medium and distributed as a program product. Alternatively, the software may be provided as a downloadable program product by an information provider connected to the Internet or other network. Such software is read from the data recording medium by an optical disk drive or other data reading device, or downloaded from server 600 or other computers via communication control module 540, and then temporarily stored in a storage module. The software is read from the storage module by processor 210 and stored in RAM in the form of an executable program. Processor 210 executes the program.

[0113] [Module detailed configuration] The module configuration of computer 200 will be described in detail with reference to Fig. 12. Fig. 12 is a block diagram showing the detailed configuration of the modules of computer 200 according to an embodiment. As shown in Fig. 12, control module 510 includes a virtual object generation module 1421, a virtual camera control module 1422, a control object control module 1423, an avatar object control module 1424, a motion detection module 1425, a collision detection module 1426, and a virtual object control module 1427.

[0114] The virtual object generation module 1421 generates data for placing various virtual objects in the virtual space 11. In one aspect, the virtual objects may include, for example, landscapes including forests, mountains, and the like, cityscapes, animals, etc., which are placed according to the progress of the game story. In another aspect, the virtual objects may include avatar objects, control objects, and stage objects.

[0115] The virtual camera control module 1422 controls the behavior of the virtual camera 14 in the virtual space 11. The virtual camera control module 1422 controls, for example, the placement position of the virtual camera 14 in the virtual space 11 and the orientation (tilt) of the virtual camera 14.

[0116] The control object control module 1423 controls a control object for receiving operations from the player 5 in the virtual space 11. The player 5 controls, for example, a virtual object placed in the virtual space 11 by operating the control object. In some situations, the control object may include, for example, a hand object (virtual hand) corresponding to the hand of the player 5 wearing the HMD 120. In some situations, the control object may correspond to the hand portion of an avatar object described below.

[0117] The avatar object control module 1424 reflects the movement of the player 5 detected by the HMD sensor 410 in the avatar object. For example, the avatar object control module 1424 detects that the player 5 has crouched and generates data for causing the avatar object to crouch. In one aspect, the avatar object control module 1424 reflects the movement of the controller 300 in the avatar object. In this case, the controller 300 includes a motion sensor, an acceleration sensor, or a plurality of light-emitting elements (e.g., infrared LEDs) for detecting the movement of the controller 300. The avatar object control module 1424 reflects the movement of the facial organs detected by the movement detection module 1425 in the face of the avatar object placed in the virtual space 11. In other words, the avatar object control module 1424 reflects the facial movement of the player 5 in the avatar object.

[0118] The movement detection module 1425 detects the movement of the player 5. For example, the movement detection module 1425 detects the movement of the hand of the player 5 in response to the output of the controller 300. For example, the movement detection module 1425 detects the movement of the body of the player 5 in response to the output of the HMD sensor 410. The movement detection module 1425 can also detect the movement of the facial organs of the player 5.

[0119] The collision detection module 1426 detects a collision when each virtual object placed in the virtual space 11 collides with another virtual object. The collision detection module 1426 can detect, for example, the timing when a virtual object comes into contact with another virtual object. The collision detection module 1426 can detect the timing when a virtual object and another virtual object cease to be in contact with each other. The collision detection module 1426 can also detect when a virtual object and another virtual object are in contact with each other. For example, when a control object and another virtual object come into contact with each other, the collision detection module 1426 detects that the control object and the other object have come into contact with each other. The collision detection module 1426 executes predetermined processing based on these detection results.

[0120] The virtual object control module 1427 controls the behavior of virtual objects excluding avatar objects in the virtual space 11. As one example, the virtual object control module 1427 transforms a virtual object. As another example, the virtual object control module 1427 changes the placement position of a virtual object. As another example, the virtual object control module 1427 moves a virtual object.

[0121] [User device hardware configuration] A user terminal (viewer terminal) 800 according to this embodiment will be described with reference to FIG. 13. FIG. 13 is a block diagram illustrating an example of a hardware configuration of the user terminal 800 according to an embodiment. The user terminal 800 (computer, information processing device) may be a mobile terminal such as a smartphone, a feature phone, a PDA (Personal Digital Assistant), or a tablet computer. The user terminal 800 may also be a gaming device suitable for playing games. As illustrated, the user terminal 800 includes a processor 810, a memory 820, a storage 830, a communication IF 13, an input / output IF 840, a touchscreen 870 (display unit), a camera 860, and a distance measurement sensor 880. These components of the user terminal 800 are electrically connected to each other via a communication bus. Note that the user terminal 800 may include an input / output IF 840 to which a display (display unit) configured separately from the user terminal 800 itself can be connected, instead of or in addition to the touchscreen 870.

[0122] 13, the user terminal 800 may be configured to be able to communicate with one or more controllers 1020. The controller 1020 establishes communication with the user terminal 800 according to a communication standard such as Bluetooth (registered trademark). The controller 1020 may have one or more buttons, etc., and transmits output values ​​based on input operations by the user 8 on the buttons, etc., to the user terminal 800. Note that the user 8 is also referred to as a viewer, as he or she is also a person who views the distributed content. The controller 1020 may also have various sensors, such as an acceleration sensor and an angular velocity sensor, and transmits output values ​​of the various sensors to the user terminal 800.

[0123] Instead of or in addition to the user terminal 800 being equipped with the camera 860 and the distance measurement sensor 880, the controller 1020 may have the camera 860 and the distance measurement sensor 880.

[0124] For example, when starting a game (or viewing), the user terminal 800 preferably has the user 8 using the controller 1020 input user identification information such as the name or login ID of the user 8 via the controller 1020. This enables the user terminal 800 to link the controller 1020 with the user 8, and based on the sender (controller 1020) of the received output value, it is possible to identify which user 8 the output value belongs to.

[0125] When a user terminal 800 communicates with multiple controllers 1020, each user 8 holds a controller 1020, allowing the single user terminal 800 to realize multiplay without communicating with other devices such as the server 600 via the network 2. Alternatively, the user terminals 800 may connect to each other via a wireless standard such as a wireless LAN standard (connecting to each other via a wireless standard without going through the server 600) to realize local multiplay using multiple user terminals 800. When the above-described multiplay is realized locally using a single user terminal 800, the user terminal 800 may further include at least some of the various functions of the server 600, which will be described later. When the above-described multiplay is realized locally using multiple user terminals 800, the multiple user terminals 800 may include the various functions of the server 600, which will be described later, in a distributed manner.

[0126] Even when the above-described multiplay is realized locally, the user terminal 800 may communicate with the server 600. For example, information indicating the results of a game, such as the score or win / loss, may be associated with user identification information and transmitted to the server 600.

[0127] The controller 1020 may also be configured to be detachable from the user terminal 800. In this case, a coupling unit for the controller 1020 may be provided on at least one surface of the housing of the user terminal 800. When the user terminal 800 and the controller 1020 are coupled to each other via the coupling unit by a wire, the user terminal 800 and the controller 1020 transmit and receive signals via the wire.

[0128] 13, the user terminal 800 may accept the attachment of a storage medium 1030 such as an external memory card via the input / output IF 840. This allows the user terminal 800 to read programs and data recorded on the storage medium 1030. The program recorded on the storage medium 1030 is, for example, a game program.

[0129] The user terminal 800 may store in the memory 820 of the user terminal 800 a game program acquired by communicating with an external device such as the server 600, or may store in the memory 820 a game program acquired by reading it from the storage medium 1030.

[0130] As described above, the user terminal 800 includes a communication IF 850, an input / output IF 840, a touch screen 870, a camera 860, a distance measurement sensor 880, and a speaker 890 as examples of mechanisms for inputting information to the user terminal 800. Each of the above-described units serving as an input mechanism can be considered as an operation unit configured to receive input operations from the user 8.

[0131] For example, when the operation unit is configured with at least one of the camera 860 and the distance measurement sensor 880, the operation unit detects an object 1010 near the user terminal 800 and identifies an input operation from the detection result of the object. As an example, the object 1010 may be a hand of the user 8, a marker of a predetermined shape, or the like, and the input operation is identified based on the color, shape, movement, or type of the object 1010 obtained as the detection result. More specifically, when the hand of the user 8 is detected from an image captured by the camera 860, the user terminal 800 identifies a gesture (a series of hand movements of the user 8) detected based on the captured image as an input operation of the user 8 and accepts it. Note that the captured image may be a still image or a video.

[0132] Alternatively, if the operation unit is configured with a touch screen 870, the user terminal 800 identifies and accepts an operation performed by the user 8 on the input unit 8701 of the touch screen 870 as an input operation by the user 8. Alternatively, if the operation unit is configured with a communication IF 850, the user terminal 800 identifies and accepts a signal (e.g., an output value) transmitted from the controller 1020 as an input operation by the user 8. Alternatively, if the operation unit is configured with an input / output IF 840, the user terminal 800 identifies and accepts a signal output from an input device (not shown) connected to the input / output IF 840 that is different from the controller 1020 as an input operation by the user 8.

[0133] The processor 810 controls the overall operation of the user terminal 800. The processor 810 includes a CPU (Central Processing Unit), an MPU (Micro Processing Unit), and a GPU (Graphics Processing Unit).

[0134] The processor 810 reads the program from the storage 830 and loads it into the memory 820. The processor 810 executes the loaded program.

[0135] The memory 820 is a main storage device. The memory 820 is composed of storage devices such as a ROM (Read Only Memory) and a RAM (Random Access Memory). The memory 820 provides a working area for the processor 810 by temporarily storing programs, virtual space data, object data, and the like that the processor 810 reads from the storage 830 (described later). The memory 820 also temporarily stores various data that the processor 810 generates while operating according to the programs.

[0136] In this embodiment, the program may be a game program for realizing a game by the user terminal 800. Alternatively, the program may be a game program for realizing the game through cooperation between the user terminal 800 and the server 600. Alternatively, the program may be a game program for realizing the game through cooperation between the user terminal 800, the server 600, and the HMD set 110. Note that the game realized through cooperation between the user terminal 800 and the server 600 and the game realized through cooperation between the user terminal 800, the server 600, and the HMD set 110 may be, for example, a game executed on a browser launched on the user terminal 800. Alternatively, the program may be a game program for realizing the game through cooperation between multiple user terminals 800. Furthermore, the various data include game-related data such as user information and game information, as well as instructions or notifications transmitted and received between devices in the HMD system 100.

[0137] The storage 830 is an auxiliary storage device. The storage 830 is configured with a storage device such as a flash memory or an HDD (Hard Disk Drive). The storage 830 stores virtual space data, object data, etc. The virtual space data, object data, etc. are provided to the user terminal 800 from the server 600 when the user terminal 800 downloads an application related to the game (including an application for viewing) for the first time, when the application related to the game (including an application for viewing) is updated, etc.

[0138] The communication IF 850 ​​controls transmission and reception of various data in the user terminal 800. The communication IF 850 ​​controls, for example, communication via a wireless LAN, internet communication via a wired LAN, a wireless LAN, or a mobile phone network, and communication using short-range wireless communication, etc.

[0139] The input / output IF 840 is an interface through which the user terminal 800 receives data input and outputs data. The input / output IF 840 may input and output data via a USB or the like. The input / output IF 840 may include, for example, physical buttons, a camera, a microphone, or a speaker of the user terminal 800.

[0140] The touch screen 870 of the user terminal 800 is an electronic component that combines an input unit 8701 and a display unit 8702. The input unit 8701 is, for example, a touch-sensitive device and is configured, for example, by a touchpad. The display unit 8702 is, for example, configured by a liquid crystal display or an organic electroluminescence (EL) display. The input unit 8701 has a function of detecting the position at which an operation by the user 8 is input on the input surface (mainly physical contact operations such as touch operations, slide operations, swipe operations, and tap operations) and transmitting information indicating the position as an input signal. The input unit 8701 may be equipped with a touch sensing unit (not shown). The touch sensing unit may be of any type, such as a capacitive type or a resistive type.

[0141] The display 430 of the HMD set 110 may be configured as a touch screen that combines an input unit and a display unit, similar to the touch screen 870 of the user terminal 800.

[0142] Although not shown, the user terminal 800 may include one or more sensors for identifying the holding orientation of the user terminal 800. The sensor may be, for example, an acceleration sensor or an angular velocity sensor. If the user terminal 800 includes a sensor, the processor 810 may identify the holding orientation of the user terminal 800 from the output of the sensor and perform processing according to the holding orientation. For example, when the user terminal 800 is held in portrait orientation, the processor 810 may select portrait screen display, in which a portrait-oriented image is displayed on the display unit 8702. On the other hand, when the user terminal 800 is held in landscape orientation, the processor 810 may select landscape screen display, in which a landscape-oriented image is displayed on the display unit. In this way, the processor 810 may be able to switch between portrait screen display and landscape screen display depending on the holding orientation of the user terminal 800.

[0143] The speaker 890 outputs sound based on the sound data. The camera 860 includes an image sensor and the like, and generates a captured image by converting incident light from a lens into an electrical signal.

[0144] Distance measurement sensor 880 is a sensor that measures the distance to a measurement object. Distance measurement sensor 880 includes, for example, a light source that emits pulse-converted light and a light-receiving element that receives the light. Distance measurement sensor 880 measures the distance to a measurement object based on the timing of light emission from the light source and the timing of light reception of reflected light that is generated when the light emitted from the light source hits the measurement object and is reflected. Distance measurement sensor 880 may also include a light source that emits directional light.

[0145] Here, an example will be further described in which the user terminal 800 receives, as an input operation from the user 8, a detection result of an object 1010 detected near the user terminal 800 using the camera 860 and the distance measurement sensor 880. The camera 860 and the distance measurement sensor 880 may be provided, for example, on the side of the housing of the user terminal 800. The distance measurement sensor 880 may be provided near the camera 860. For example, an infrared camera may be used as the camera 860. In this case, the camera 860 may be provided with a lighting device that irradiates infrared light, a filter that blocks visible light, and the like. This can further improve the accuracy of object detection based on images captured by the camera 860, regardless of whether the location is indoors or outdoors.

[0146] The processor 810 may perform one or more of the following processes (1) to (5) on the image captured by the camera 860. (1) The processor 810 performs image recognition processing on the image captured by the camera 860 to determine whether the hand of the user 8 is included in the captured image. The processor 810 may use, for example, a technique such as pattern matching as the analysis technique employed in the image recognition processing. (2) The processor 810 also detects a gesture by the user 8 from the shape of the hand of the user 8. For example, the processor 810 determines the number of fingers of the user 8 (the number of outstretched fingers) from the shape of the hand of the user 8 detected from the captured image. The processor 810 further determines the gesture performed by the user 8 from the determined number of fingers. For example, if the number of fingers is five, the processor 810 determines that the user 8 has performed the "paper" gesture. Furthermore, if the number of fingers is zero (no fingers detected), the processor 810 determines that the user 8 has made a "rock" gesture. Furthermore, if the number of fingers is two, the processor 810 determines that the user 8 has made a "scissors" gesture. (3) The processor 810 performs image recognition processing on the image captured by the camera 860, thereby detecting whether the user 8 has only the index finger extended or whether the user 8 has made a flicking motion. (4) The processor 810 detects the distance between the user terminal 800 and an object 1010 (such as the hand of the user 8) near the user terminal 800, based on at least one of the image recognition results of the image captured by the camera 860 and the output value of the distance measurement sensor 880, etc. For example, processor 810 detects whether the hand of user 8 is near (for example, at a distance less than a predetermined value) or far away (for example, at a distance equal to or greater than a predetermined value) from user terminal 800 based on the size of the shape of the hand of user 8 identified from the image captured by camera 860. Note that if the captured image is a video, processor 810 may detect whether the hand of user 8 is approaching or moving away from user terminal 800.(5) When it is determined that the distance between the user terminal 800 and the hand of the user 8 is changing while the hand of the user 8 is detected based on the image recognition result of the image captured by the camera 860, the processor 810 recognizes that the user 8 is waving his / her hand in the imaging direction of the camera 860. When the distance measuring sensor 880, which has a stronger directionality than the imaging range of the camera 860, detects and does not detect an object, the processor 810 recognizes that the user 8 is waving his / her hand in a direction perpendicular to the imaging direction of the camera.

[0147] In this way, the processor 810 detects whether the user 8 is clenching their hand (whether it is a "rock" gesture or another gesture (e.g., "paper")) by performing image recognition on the image captured by the camera 860. The processor 810 also detects how the user 8 is moving their hand, along with the shape of the user 8's hand. The processor 810 also detects whether the user 8 is moving their hand closer to or farther away from the user terminal 800. Such an operation can correspond to an operation using a pointing device such as a mouse or a touch panel, for example. The user terminal 800, for example, moves a pointer on the touch screen 870 in accordance with the movement of the user 8's hand, and detects the "rock" gesture of the user 8. In this case, the user terminal 800 recognizes that the user 8 is continuing a selection operation. The continuation of the selection operation corresponds, for example, to the mouse being clicked and maintained in a pressed state, or to the touch panel being touched and maintained in a touched state after a touch-down operation. Furthermore, when the user 8 moves his / her hand further while the user terminal 800 has detected the "rock" gesture of the user 8, the user terminal 800 can recognize such a series of gestures as an operation corresponding to a swipe operation (or a drag operation). Furthermore, when the user terminal 800 detects a gesture of the user 8 snapping his / her fingers based on the detection result of the hand of the user 8 from the image captured by the camera 860, the user terminal 800 may recognize the gesture as an operation corresponding to a mouse click or a tap operation on a touch panel.

[0148] <Virtual space in user terminal 800> FIG. 14 is a diagram schematically illustrating an example of a virtual space 11 and a field of view image 1400 defined in a user terminal 800. In FIG. 14(A), an avatar object 6 and other objects A to D are arranged in the virtual space 11. The computer 200 distributes, to the user terminal 800, action instruction data (described later) for specifying the positions and actions of the avatar object 6 and the other objects A to D in the virtual space 11 shown in FIG. 8. Therefore, the positions and movements of the avatar object 6 and the other objects A to D in the virtual space 11 shown in FIG. 14 match the positions and movements of the avatar object 6 and the other objects A to D in the virtual space 11 shown in FIG. 8. A user 8A holds a user terminal 800A in his left hand. In the example of FIG. 14, the user 8A watches a live game progress part, which is a program performed by the avatar object 6, while viewing the screen of the user terminal 800A.

[0149] In the virtual space 11 defined by the user terminal 800, the avatar object 6 moves in the same manner as the avatar object 6 in the virtual space 11 shown in FIG. 8 based on the action instruction data. User 8A watches the performance of the avatar object 6 as a participant in the live game progression part through the screen of the user terminal 800A. Other users 8C and 8D watch the performance of the avatar object 6 as participants in the live game progression part through the screens of other user terminals 800C and 800D. In this way, the live game progression part progressed by the avatar object 6 is distributed to multiple users 8 simultaneously.

[0150] In the example of FIG. 14(A), in the virtual space 11 defined by the user terminal 800A, a virtual camera 20A is arranged in front of the avatar object 6. The virtual camera 20A defines a viewing area 21 corresponding to the position and orientation of the virtual camera 20A. The processor 810 generates a viewing image 1400 corresponding to the viewing area 21 and displays it on the display unit 8702 of the touch screen 870 as shown in FIG. 14(B). That is, the viewing area 21 is defined from the viewpoint of the virtual camera 20A, and the viewing image 1400 is also displayed from the viewpoint of the virtual camera 20A. The viewing image 1400 at least includes the avatar object 6 that performs the performance. The user 8A visually recognizes the avatar object 6 and a part of the virtual space 11 where the avatar object 6 appears by visually recognizing the viewing image 1400. Thereby, the user 8A can obtain a virtual experience as if the avatar object 6 were an actual distributor.

[0151] <Functional Configuration of the HMD System 100> FIG. 15 is a block diagram showing the functional configurations of the HMD set 110, the user terminal 800, and the server 600 included in the HMD system 100. Each of the HMD set 110, the user terminal 800, and the server 600 may include a functional configuration necessary when functioning as a general computer (not shown) and a functional configuration necessary for realizing known functions in a game.

[0152] The HMD set 110 has a function of arranging and operating objects in the virtual space 11 shown in Fig. 8, generating action instruction data for arranging and operating the objects in the virtual space 11 shown in Fig. 14, and supplying the generated data to the user terminal 800. The action instruction data may be data for enabling the user terminal 800 to display gameplay footage in the live game progression part progressed by the avatar object 6, and may include, for example, motion capture data (also referred to as motion data) of the movements of the player 5 (e.g., a model) input via the motion sensor 420, audio data recording the voice of the player 5 (e.g., a voice actor), operation history data indicating a history of input operations for operating other objects by the player 5, commands associated with a series of input operations, or a group of motion commands in chronological order that arrange commands for arranging or moving objects in the virtual space 11 based on a game program. The HMD set 110 functions as a control unit 2100 (control module 510, rendering module 520) and a storage unit 2200 (memory module 530) through cooperation of the processor 210, memory 220, storage 230, communication IF 250, input / output IF 240, and the like.

[0153] The user terminal 800 functions as an input device that accepts input operations from the user 8, and as an output device that outputs game images and sounds. The user terminal 800 functions as a control unit 8100 and a storage unit 8200 through cooperation of a processor 810, a memory 820, a storage 830, a communication IF 850, an input / output IF 840, and the like.

[0154] The server 600 has a function of communicating with each user terminal 800 and supporting viewing of game images on the user terminal 800. For example, when the user terminal 800 downloads an application relating to the game for the first time or updates it, the server 600 provides the user terminal 800 with data to be stored in the user terminal 800 in order to display gameplay footage, and data of objects that may be included in the gameplay footage (also referred to as object display data or object data).

[0155] The server 600 has a function of acting as an intermediary between the user terminal 800 and the HMD set 110. This enables the HMD set 110 to provide action instruction data to the user terminal 800 or a group of multiple user terminals 800 in a timely manner without misplacing the destination. The server 600 functions as a control unit 6100 and a storage unit 6200 through cooperation of a processor 610, a memory 620, a storage 630, a communication IF 650, an input / output IF 640, and the like.

[0156] If the game is a multiplayer game, the server 600 may have a function of communicating with each HMD set 110 participating in the game, and a function of mediating interactions between the HMD sets 110 and a synchronization control function.

[0157] The memory unit 2200 stores a game program 231. The memory unit 8200 stores a game program 831. The memory unit 6200 stores a game program 631. The game program 231 is a game program executed by the HMD set 110. The game program 831 is a game program executed by the user terminal 800. The game program 631 is a game program executed by the server 600. The devices cooperate to operate based on the game programs 231, 631, and 831, thereby realizing a game related to the HMD system 100. Note that the game programs 231 and 831 may be stored in the memory unit 6200 and downloaded to the memories 2200 and 8200, respectively.

[0158] Furthermore, the memories 2200, 8200, and 6200 store game information 132 and user information 133, respectively. The game information 132 is data that the control units 2100, 8100, and 6100 refer to when executing the game programs 231, 831, and 631, respectively. The user information 133 is data related to the account of the user 8. In the memories 2200 and 6200, the game information 132 and the user information 133 are stored for each user terminal 800. Note that the user information 133 may include a purchase history of virtual currency (chargeable items), and a purchase or acquisition history of specific items.

[0159] (Functional configuration of HMD set 110) The control unit 2100 executes a game program 231 stored in the storage unit 2200 to comprehensively control the HMD set 110. For example, the control unit 2100 places and moves objects in the virtual space 11 shown in FIG. 8 in accordance with the game program 231 and the operation of the player 5, and generates action instruction data associated with the placement and movement and supplies the data to the user terminal 800. The control unit 2100 may further execute the game program 231 as necessary. The control unit 2100 also communicates with the server 600 and the user terminal 800 that is running the game to send and receive information.

[0160] The control unit 2100 functions as an operation reception unit 2101, a display control unit 2102, a UI control unit 2103, an animation generation unit 2104, a game progression unit 2105, a virtual space control unit 2106, and a reaction processing unit 2107 in accordance with the description of the game program 231. The control unit 2100 can also function as other functional blocks (not shown) in order to control objects appearing in a game in accordance with the nature of the game executed in the HMD system 100.

[0161] The operation reception unit 2101 detects and receives input operations from the player 5. The input operations are input via the input / output IF 34 from various input devices, such as the HMD 120, the motion sensor 420, an input unit formed integrally with the display unit when the display 430 is a touch screen, a mouse (not shown), a keyboard (not shown), etc. The operation reception unit 2101 determines what kind of input operation has been performed from the action performed by the player 5, and outputs the result to each element of the control unit 2100.

[0162] A user interface (hereinafter referred to as UI) control unit 2103 controls the UI image displayed on the HMD 120 or the display 430.

[0163] The animation generation unit 2104 generates animations showing the motions of various objects based on the control modes of the various objects. For example, the animation generation unit 2104 may generate animations that reproduce the game screens actually displayed on the user terminal 800 that is the communication partner.

[0164] The display control unit 2102 outputs to the HMD 120 or the display 430 an image that reflects the results of the processing performed by the above-mentioned elements.

[0165] The game progression unit 2105 progresses the game based on the avatar object 6 in response to input operations by the player 5 and operations by the user 8 of the user terminal 800. For example, the game progression unit 2105 performs predetermined game processing when the avatar object 6 performs a predetermined action. Furthermore, for example, the game progression unit 2105 receives information representing operations by the user 8 on the user terminal 800, and performs game processing based on the operations by the user 8. The game progression unit 2105 updates various game parameters based on the game processing. Game information including the game parameters is supplied to the user terminal 800 via the server 600.

[0166] The virtual space control unit 2106 includes modules 1421 to 1427 of the control module 510 described with reference to Fig. 12. The details of these modules have already been described.

[0167] The virtual space control unit 2106 performs various controls for sharing the virtual space 11 with the user terminal 800. FIG. 16 is a diagram showing a detailed configuration of the virtual space control unit 2106. As shown in FIG. 16, the virtual space control unit 2106 includes modules 1421 to 1427 described with reference to FIG. 12. The virtual space control unit 2106 controls the movement of the avatar object 6 and the behavior of virtual objects in the HMD set 110 through the operation of these modules 1421 to 1427. Furthermore, the virtual space control unit 2106 generates action instruction data for moving the avatar object 6 in the user terminal 800 and action instruction data for controlling the behavior of each virtual object, based on the operation of the modules 1421 to 1427. This action instruction data is supplied to the user terminal 800 via the server 600. For example, the virtual space control unit 2106 generates action instruction data for instructing the controlled object to speak, based on voice data input by the player 5 (such as a voice actor) via the microphone 170. The action instruction data generated in this manner includes at least the audio data described above. For example, the virtual space control unit 2106 generates action instruction data instructing the controlled object to perform a movement based on motion capture data input by the player 5 (e.g., a model) via the motion sensor 420. The action instruction data generated in this manner includes at least the motion capture data described above. For example, the virtual space control unit 2106 generates action instruction data instructing the controlled object to perform a movement based on operation history data, i.e., a history of input operations input by the player 5 via an input mechanism such as the controller 300 or an operation unit such as a touch screen, mouse, or keyboard. The action instruction data generated in this manner includes at least the operation history data described above. The operation history data is, for example, information in which an operation log indicating which button on the controller 300 the player 5 pressed and at what timing when a specific screen was displayed on the HMD 120 is organized in chronological order. Alternatively, the virtual space control unit 2106 identifies a command instructing the object to perform a movement, associated with an input operation input by the player 5 via the input mechanism or operation unit described above.The virtual space control unit 2106 may then arrange the commands in the order they were input to generate a group of motion commands that indicate a series of movements of the object, and generate action instruction data that instructs the object to move in accordance with the group of motion commands. The action instruction data generated in this manner includes at least the group of motion commands described above.

[0168] The reaction processing unit 2107 receives feedback on the reaction of the user 8 from the user terminal 800 and outputs this to the player 5 of the HMD set 110. In this embodiment, for example, while the user terminal 800 is operating an object in accordance with the above-mentioned action instruction data, the user 8 can create a comment addressed to the object. The reaction processing unit 2107 receives comment data of the comment and outputs it. The reaction processing unit 2107 may display text data corresponding to the comment of the user 8 on the display unit 352, or may output audio data corresponding to the comment of the user 8 from a speaker (not shown).

[0169] (Functional configuration of server 600) The control unit 6100 executes a game program 631 stored in the storage unit 6200 to exercise overall control over the server 600. For example, the control unit 6100 transmits various data, programs, and the like to the user terminal 800. The control unit 6100 receives some or all of the game information or user information from the user terminal 800. If the game is a multiplayer game, the control unit 6100 may receive a request for multiplayer synchronization from the HMD set 110 and transmit data for synchronization to the HMD set 110. Furthermore, the control unit 6100 communicates with the user terminal 800 and the HMD set 110 as necessary to transmit and receive information.

[0170] The control unit 6100 functions as a progress support unit 6101 and a sharing support unit 6102 in accordance with the description of the game program 631. The control unit 6100 can also function as other functional blocks (not shown) to support the progress of the game on the user terminal 800 in accordance with the nature of the game being executed.

[0171] The progress support unit 6101 communicates with the user terminal 800 and supports the user terminal 800 in progressing through various parts included in the game. For example, when the user terminal 800 progresses through the game, the progress support unit 6101 provides the user terminal 800 with information necessary for progressing through the game.

[0172] The sharing support unit 6102 communicates with multiple user terminals 800 and HMD sets 110, and supports the progress of the game and the sharing of the virtual space 11 between these devices. The sharing support unit 6102 may also have a function of matching online user terminals 800 and HMD sets 110. This allows smooth transmission and reception of information between the user terminals 800 and the HMD sets 110.

[0173] (Functional configuration of user terminal 800) The control unit 8100 executes a game program 831 stored in the storage unit 8200 to exercise overall control over the user terminal 800. For example, the control unit 8100 supports the progress of the game by the player 5 in accordance with the game program 831 and operations by the user 8. Furthermore, while the game is being played by the player 5, the control unit 8100 communicates with the HMD set 110 via the server 600 as necessary to send and receive information.

[0174] The control unit 8100 functions as an operation reception unit 8101, a display control unit 8102, a UI control unit 113, an animation generation unit 8104, a game progression unit 8105, a virtual space control unit 8106, and a progress information generation unit 8107 in accordance with the description of the game program 831. The control unit 8100 can also function as other functional blocks (not shown) in order to progress the game in accordance with the nature of the game to be executed.

[0175] The operation reception unit 8101 detects and receives an input operation by the user 8 to the input unit 8701. The operation reception unit 8101 determines what kind of input operation has been performed from the action performed by the user 8 on the touch screen 870 and the console via the other input / output IF 14, and outputs the result to each element of the control unit 8100. The detailed function of the operation reception unit 8101 is almost the same as that of the operation reception unit 2101 in the HMD set 110.

[0176] For example, the operation reception unit 8101 receives an input operation on the input unit 8701, detects the coordinates of the input position of the input operation, and identifies the type of the input operation. The operation reception unit 8101 identifies, for example, a touch operation, a slide operation, a swipe operation, a tap operation, etc. When a continuously detected input is interrupted, the operation reception unit 8101 detects that the touch input from the touch screen 870 has been released.

[0177] The UI control unit 8103 controls UI images to be displayed on the display unit 8702 in order to construct a UI. The UI images are tools that allow the user 8 to input necessary information for the progress of the game to the user terminal 800, or to obtain information output during the progress of the game from the user terminal 800. The UI images are not limited to these, but include, for example, icons, buttons, lists, menu screens, etc.

[0178] The animation generation unit 8104 generates animations showing the motion of various objects based on the control modes of the various objects. For example, the animation generation unit 8104 may generate animations that show the objects moving as if they were actually there, moving their mouths, changing their facial expressions, etc.

[0179] The display control unit 8102 outputs a game screen reflecting the results of the processing executed by the above-mentioned elements to the display unit 8702 of the touch screen 870. The display control unit 8102 may display a game screen including animation generated by the animation generation unit 8104 on the display unit 8702. The display control unit 8102 may also draw the above-mentioned UI image controlled by the UI control unit 8103, superimposed on the game screen. Furthermore, the display control unit 8102 may draw various game parameters, superimposed on the game screen, based on game information supplied from the HMD set 110.

[0180] The virtual space control unit 8106 performs various controls for sharing the virtual space 11 with the HMD set 110. Fig. 17 is a diagram showing the detailed configuration of the virtual space control unit 8106. As shown in Fig. 17, the virtual space control unit 8106 includes a virtual object generation module 8121, a virtual camera control module 8122, an avatar object control module 8124, and a virtual object control module 8127.

[0181] The virtual object generation module 8121 is configured in the same manner as the virtual object generation module 1421 described with reference to FIG. 12 . Here, virtual objects generated in the user terminal 800 are synchronized with virtual objects generated in the HMD set 110. However, whether or not each virtual object is synchronized may be determined for each virtual object. For example, some virtual objects generated in the HMD set 110 may not be generated in the user terminal 800. Furthermore, some virtual objects generated in the HMD set 110 may be generated in a different display form in the user terminal 800. The display form may be, for example, color, transparency, or the like, but is not limited to these. Furthermore, the virtual space 11 generated in the user terminal 800 may include virtual objects other than the virtual objects generated in the HMD set 110.

[0182] The virtual camera control module 8122 is configured in the same manner as the virtual camera control module 1422 described with reference to Fig. 12. Here, the virtual cameras 20A, 20C, and 20D arranged in the virtual spaces 11A, 11C, and 11D realized in the user terminals 800A, 800C, and 800D are different from the virtual camera 14 arranged in the virtual space 11 realized in the HMD set 110. In other words, the field of view image of the virtual space 11 shared between the HMD set 110 and the user terminal 800 may differ between the HMD set 110 and the user terminal 800.

[0183] In user terminals 800A, 800C, and 800D, the positions and directions of virtual cameras 20A, 20C, and 20D may be operable or inoperable by users 8A, 8C, and 8D. Furthermore, if virtual cameras 20A, 20C, and 20D are inoperable, the positions and directions of virtual cameras 20A, 20C, and 20D may be the same. In this case, multiple users 8 view a common field of view image through each user terminal 800. In other words, in this case, multiple users 8 can view a field of view image (live video) captured by a common virtual camera.

[0184] In this embodiment, the positions and directions of virtual cameras 20A, 20C, and 20D can be controlled by users 8A, 8C, and 8D, and an example will be described in which multiple users 8 view different field-of-view images. In one aspect, the position and direction of virtual camera 20A change in response to a slide operation (or swipe operation) on the touch screen of user terminal 800A. For example, the position of virtual camera 20A moves left in response to a leftward slide operation, moves right in response to a rightward slide operation, moves upward in response to an upward slide operation, and moves downward in response to a downward slide operation. At this time, the orientation of virtual camera 20A is adjusted, for example, so that avatar object 6 fits within field-of-view area 21 (specifically, so that avatar object 6 is at the center of field-of-view area 21).

[0185] 12 , except that the avatar object control module 8124 operates in response to action instruction data instead of operating in response to the movement of the HMD 120 or the movement of the controller 300. The action instruction data is generated in the HMD set 110 in response to the movement of the HMD 120 or the movement of the controller 300, and is delivered to the user terminal 800.

[0186] The virtual object control module 8127 is configured in the same way as the virtual object control module 1427 described with reference to Fig. 12, except that it operates in accordance with action instruction data. The action instruction data is generated in the HMD set 110 so as to represent the behavior of virtual objects other than the avatar object 6 in the HMD set 110, and is delivered to the user terminal 800.

[0187] In this way, the virtual space control unit 8106 analyzes (renders) the action instruction data and moves the avatar object 6 and other virtual objects based on the analysis results. In this embodiment, the virtual space control unit 8106 is triggered by receiving the action instruction data supplied by the HMD set 110 via the communication IF 33, and moves the avatar object 6 or other virtual objects based on the data. This makes it possible for the user 8 to see the avatar object 6 and other virtual objects moving in real time.

[0188] The game progression unit 8105 supports the progress of the game by the player 5 of the HMD set 110, in accordance with input operations by the user 8 input via the operation reception unit 8101. For example, the game progression unit 8105 supports the progress of the game by the player 5 by placing item objects in the virtual space 11 while the game is progressing.

[0189] The progress information generation unit 8107 generates progress information that indicates the progress of the game support being performed by the game progression unit 8105, and transmits this information to the server 600 or the HMD set 110 as appropriate. The progress information may include, for example, information specifying the currently displayed game screen, or may include a progress log that indicates the progress of the game in chronological order using characters, symbols, etc. In an embodiment of the HMD system 100 in which the server 600 and the HMD set 110 do not require progress information, the progress information generation unit 8017 may be omitted.

[0190] 15 are merely examples. Each of the HMD set 110, the server 600, and the user terminal 800 may have at least some of the functions of the other devices. Furthermore, a device other than the HMD set 110, the server 600, and the user terminal 800 may be a component of the HMD system 100, and the other device may execute some of the processing in the HMD system 100. That is, in this embodiment, the computer that executes the game program may be any of the HMD set 110, the server 600, the user terminal 800, or another device, or may be realized by a combination of these multiple devices.

[0191] 15 is merely an example. The server 600 may have at least some of the functions of the user terminal 800. The user terminal 800 may have at least some of the functions of the server 600. Furthermore, devices other than the user terminal 800 and the server 600 may be components of the HMD system 100, and some of the processing in the HMD system 100 may be executed by these other devices. That is, in this embodiment, the computer that executes the game program may be any of the user terminal 800, the server 600, and other devices, or may be realized by a combination of these multiple devices.

[0192] <Game Overview> 18(A) to 18(D), a game executed by the HMD system 100 according to this embodiment (hereinafter referred to as the present game) is, for example, a game in which an avatar object 6 operating a weapon such as a gun or a knife and a plurality of enemy objects 23 appear in a virtual space 11, and the avatar object 6 fights against the enemy objects 23. Various game parameters such as the stamina of the avatar object 6, the number of usable magazines, the number of remaining bullets in the gun, and the number of remaining enemy objects 23 are updated as the game progresses.

[0193] The game includes a plurality of stages, and the player 5 can clear each stage by fulfilling a predetermined achievement condition associated with that stage. The predetermined achievement condition may include, for example, a condition that is fulfilled by defeating all enemy objects 23 that appear, defeating a boss object among the enemy objects 23 that appear, obtaining a predetermined item, reaching a predetermined position, etc.

[0194] In this game, in order to share the virtual space 11 between the HMD set 110 and multiple user terminals 800, game information including the various game parameters described above and operation instruction data for arranging, operating, etc. objects are live-streamed to multiple user terminals 800 at predetermined time intervals.

[0195] As a result, a field of view image 1400 of a field of view area 21 in the virtual space 11 defined by the virtual camera 20 corresponding to the user terminal 800 is displayed on the touch screen 870 of the user terminal 800 while the user is watching the game. In addition, parameter images representing the avatar object 6's stamina, the number of available magazines, the number of bullets remaining in the gun, the number of enemy objects 23 remaining, etc. are superimposed and displayed on the upper right and upper left sections of the field of view image 1400.

[0196] Details of the action instruction data are as follows: That is, during live distribution, the computer 200 of the HMD set 110 generates action instruction data for controlling the actions of the avatar object 6 in response to the operation of the player 5, action instruction data for controlling the placement and actions of other objects such as the enemy object 23 in accordance with the game program, and so on, and distributes the action instruction data to the plurality of user terminals 800 via the server 600. Each of the plurality of user terminals 800 analyzes (renders) the action instruction data upon receiving the action instruction data.

[0197] In other words, the operation instruction data includes information for identifying the type of object to be placed, the position, posture, orientation, etc. of the object, and the user terminal 800 analyzes the operation instruction data to identify which object to place at which position in the virtual space 11, and in what posture and orientation, etc.

[0198] The memory 820 of the user terminal 800 stores object data such as the avatar object 6 and the enemy object 23 in advance. The user terminal 800 uses the object data to place an object of a type corresponding to the analysis result of the action instruction data at a position in the virtual space 11 corresponding to the analysis result, with a posture, orientation, etc. corresponding to the analysis result. The placement and movement of the avatar object 6, the enemy object 23, etc. in the virtual space 11 defined by the user terminal 800 coincide with the placement and movement of the avatar object 6, the enemy object 23, etc. in the virtual space 11 defined by the computer 200.

[0199] The computer 200 of the HMD set 110 varies the level design corresponding to the difficulty level of the game depending on the number of user terminals 800 (number of viewers) currently viewing the game. That is, the computer 200 adjusts the difficulty level for achieving the achievement conditions associated with each of a plurality of stages depending on the number of viewers at the start of that stage. In some situations, the computer 200 increases the difficulty level of the game to prevent the player 5 from easily clearing the game, since a large number of viewers can be expected to provide more support from the viewers, as will be described later.

[0200] The difficulty level is determined, for example, based on the combat capabilities and appearance numbers of enemy objects 23 and the combat capabilities of avatar objects 6. In one aspect, when the number of viewers is zero, computer 200 sets the combat capabilities and appearance numbers of enemy objects 23 and the combat capabilities of avatar objects 6 to initial values, and as the number of viewers increases, computer 200 may increase the combat capabilities and appearance numbers of enemy objects 23 from the initial values ​​or decrease the combat capabilities of avatar objects 6 from the initial value. The number of viewers for which the initial values ​​are set may not be zero, but may be any number other than zero.

[0201] In a certain situation, the combat ability of the enemy object 23 is determined by the movement speed (agility) and stamina of the enemy object 23, and the number of enemy objects 23 that appear is determined by the total number of enemy objects 23 that appear in the current stage and their appearance frequency. Also, the combat ability of the avatar object 6 is determined by the movement speed (agility) and stamina of the avatar object 6, and correction values ​​depending on the types of weapons and armor equipped.

[0202] On the touch screen 870 of the user terminal 800, a UI image 22 for receiving an item insertion operation from the user 8 to support the avatar object 6 is displayed superimposed on an image representing the virtual space 11. The UI image 22 depicts icons of, for example, a magazine, a first aid kit, and an obstacle (such as a traffic cone or a barricade), and the item insertion operation corresponds to tapping on any of the icons.

[0203] When the user 8 performs the item insertion operation, the user terminal 800 consumes virtual currency (value item) that the user 8 has purchased and owns, and places a predetermined effect object 24 (see FIG. 18(C)) such as a gift box in the virtual space 11. The effect using the effect object 24 (hereinafter also referred to as the insertion effect) is executed for a predetermined time (e.g., 2 seconds). The insertion effect may, for example, display a countdown at the same time as placing the gift box, and open the lid of the gift box when the count value reaches 0. The size, shape, color, etc. of the gift box may be the same regardless of the type of item inserted, or may differ depending on the type of item and the amount of virtual currency consumed. The virtual currency is associated with the user 8, for example, by paying a fee or completing a quest in the game. The virtual currency may be paid for or free of charge.

[0204] 19, the user terminal 800 transmits item insertion information to the computer 200 via the server 600 in response to the item insertion operation. The item insertion information includes information for identifying the type of item specified by the item insertion operation and information for identifying the timing at which the computer 200 will place the item in the virtual space 11 or activate a support effect (support action) by making the item available for use in the virtual space 11. The activation timing of the support effect is set to coincide with the timing at which the effect by the effect object 24 ends (i.e., the timing at which the predetermined time has elapsed since the effect object 24 was placed in the virtual space 11). Note that the support effect of the item may be activated immediately or after a certain time has elapsed after the item is inserted, without performing any effect.

[0205] If the item to be dropped is an obstacle, the user terminal 800 specifies a part of the range within the field of view of the virtual camera 20 as the item drop range. At this time, the item drop information includes information for specifying the item drop range. The item drop range is, for example, the range around the avatar object 6.

[0206] When the computer 200 receives item input information, it places an item corresponding to the item type identified from the item input information in the virtual space 11 or makes it available for use within the virtual space 11 at the activation timing identified from the item input information (more specifically, at the timing when the specified time has elapsed since the item input information was received) in order to support the avatar object 6.

[0207] As a result, if the item type is a magazine, the computer 200 restores the number of remaining bullets in the gun used by the avatar object 6 at the activation timing. If the item type is a first aid kit, the computer 200 restores the stamina of the avatar object 6 at the activation timing. If the item type is an obstacle, the computer 200 identifies any position within the item insertion range identified from the item insertion information as the item insertion position, and places an object representing the obstacle at the item insertion position at the activation timing. An obstacle is, for example, an item that has the function of hindering the progress or attacks of the enemy object 23. Therefore, an obstacle can be said to be an item that supports the avatar object 6 by placing the obstacle.

[0208] The computer 200 adjusts the degree (magnitude) of the support effect depending on the number of viewers at the start of the current stage. In one aspect, when the number of viewers is zero, the computer 200 sets the degree of the support effect to an initial value, and decreases the degree of the support effect as the number of viewers increases. The degree of the support effect is determined by the amount of recovery for the number of remaining bullets in the gun used by the avatar object 6 and the stamina of the avatar object 6, and by the size, strength, weight, etc. of obstacles.

[0209] Therefore, when the level of the support effect increases, the recovery amount of the remaining number of bullets in the gun or the recovery amount of the avatar object 6's stamina increases, and the size, strength, weight, etc. of the obstacle increases. An increase in the recovery amount of the remaining number of bullets in the gun or the recovery amount of the avatar object 6's stamina extends the battle time of the avatar object 6, and an increase in the size, strength, weight, etc. of the obstacle extends the time it takes for the enemy object 23 to reach the avatar object 6.

[0210] When the inserted item is an obstacle, the computer 200 generates action instruction data including information for identifying the type of obstacle object and the position and posture of the object, and distributes the action instruction data to multiple user terminals 800 via the server 600 (see Figure 19).

[0211] Each user terminal 800 analyzes the action instruction data to identify the type of item object to be placed, as well as the position and orientation of the item object. Furthermore, each user terminal 800 places the item object in the virtual space 11 based on the analysis results and using object data pre-stored in memory 820.

[0212] In this way, object data is stored in advance on the user terminal 800 side, and when placing an object, action instruction data, which has a much smaller data volume than video data, is transmitted to the multiple user terminals 800. As a result, the time from when an item is inserted until the item object is displayed on the user terminal 800 can be shortened, and the timing of placing the item object can be synchronized among the multiple user terminals 800. Furthermore, in the user terminal 800 that performed the item insertion operation, the item object is placed in the virtual space 11 at the same time as the effect by the effect object 24 ends.

[0213] (About operation) 20 and 21 are flowcharts showing an example of the basic game progression of the game. The processing according to the flowchart shown in Fig. 20 is executed by cooperation between an operation reception unit 8101, a display control unit 8102, a UI control unit 8103, an animation generation unit 8104, a game progression unit 8105, a virtual space control unit 8106, and a progress information generation unit 8107 provided in a control unit 8100 of the user terminal 800. The processing according to the flowchart shown in Fig. 21 is executed every predetermined time (e.g., 1 / 60 seconds) by cooperation between an operation reception unit 2101, a display control unit 2102, a UI control unit 2103, an animation generation unit 2104, a game progression unit 2105, a virtual space control unit 2106, and a reaction processing unit 2107 provided in a control unit 2100 of the computer 200.

[0214] 20 and 21 may be executed by the server 600, and the processing results may be transmitted to the user terminal 800 or the computer 200. Furthermore, transmission and reception of information between the user terminal 800 and the computer 200 is performed via the server 600, but this is not limiting and the transmission and reception may be performed without the server 600.

[0215] 20, in step S01, it is determined whether or not a viewing start operation has been performed by user 8 based on an operation input from touch screen 870. If it is determined that a viewing start operation has been performed, the process proceeds to step S02, where viewing start information is transmitted to computer 200. In step S03, virtual space 11 is defined based on virtual space data stored in storage 830. In step S04, virtual camera 20 is placed at a predetermined initial position in virtual space 11. The initial position of virtual camera 20 is, for example, a position that captures center 12 of virtual space 11 shown in FIG. 4 from diagonally above.

[0216] In step S05, the action instruction data and game information distributed from computer 200 are received. In step S06, the action instruction data is analyzed to determine the positions and postures of virtual objects such as avatar object 6 and enemy object 23. In step S06, based on the analysis results and the object data stored in memory 820, virtual objects such as avatar object 6 and enemy object 23 are placed in virtual space 11, or virtual objects already placed are caused to move. In step S06, various game parameter values ​​such as the stamina of avatar object 6, the number of available magazines, the number of remaining bullets in the gun, and the number of remaining enemy objects 23 are determined based on the game information.

[0217] As will be described later, the action instruction data and game information are transmitted from computer 200 at predetermined intervals (1 / 60 seconds). Therefore, in step S06, objects in virtual space 11 are updated at the predetermined intervals based on the received action instruction data, and various game parameter values ​​are updated at the predetermined intervals based on the received game information. Note that the objects placed in virtual space 11 also include item objects inserted by user 8.

[0218] In step S07, a field of view 21 is defined according to the current position and orientation of virtual camera 20, and a field of view image 1400 corresponding to field of view 21 is generated. In step S08, a UI image 22 for receiving an item insertion operation from user 8 and a parameter image representing the above-mentioned various game parameter values ​​are superimposed on field of view image 1400 and displayed on touch screen 870 (see FIG. 18). As a result, gameplay footage of the game is displayed on touch screen 870.

[0219] The UI image 22 depicts icons and the like for receiving an item insertion operation from the user 8. In one situation, the items include a gun magazine used by the avatar object 6, a first aid kit for restoring the stamina of the avatar object 6, and obstacles such as traffic cones and barricades for blocking attacks by the enemy object 23.

[0220] Each item is associated with an amount of virtual currency required to insert the item and points to support the avatar object 6. The points associated with an item are set to a value corresponding to the amount of virtual currency required to insert the item, and for example, the greater the amount of virtual currency required to insert the item, the greater the points.

[0221] In addition, parameter images representing the avatar object 6's stamina, the number of available magazines, and the number of remaining bullets in the gun are superimposed on the upper right corner of the field of view image 1400, and a parameter image representing the number of remaining bullets in the enemy object 23 is superimposed on the upper left corner of the field of view image 1400.

[0222] In step S09, it is determined whether or not an item insertion operation has been performed on the UI image 22 based on an operation input from the touch screen 870. If it is determined that the item insertion operation has been performed, the process proceeds to step S10.

[0223] In step S10, virtual currency corresponding to the inserted item is consumed from the virtual currency owned by the user 8. That is, when an item insertion operation is performed, the amount of virtual currency consumed corresponds to the item.

[0224] If the item specified by the item insertion operation is an obstacle, in step S11, a portion of the range within the field of view image 1400 is identified as the item insertion range based on the position and orientation (viewpoint) of virtual camera 20. For example, as shown in FIG. 18(B), in a situation where virtual camera 20 is capturing virtual space 11 at a wide angle from the right side of avatar object 6, the floor surface in front of avatar object 6 is identified as the item insertion range. Once the processing of step S11 is completed, the process proceeds to step S12. On the other hand, if the item specified by the item insertion operation is a magazine or a first aid kit, the process proceeds to step S12 without executing the processing of step S11.

[0225] In step S12, item insertion information is sent to computer 200. The item insertion information includes the type of item specified by the item insertion operation, the timing at which the support effect of the item is activated, the amount of points associated with the item, and information for identifying the ID of user 8 who performed the item insertion operation. Furthermore, when the processing of step S11 is executed, the item insertion information also includes information for identifying the item insertion range (the viewpoint of virtual camera 20). Note that when the amount of points associated with the item is managed by computer 200, the amount of points does not need to be included in the item insertion information.

[0226] In step S13, a predetermined effect object 24 different from the inserted item (not related to the inserted item) is placed in the virtual space 11. The effect object 24 is placed, for example, between the position of the virtual camera 20 and the position of the avatar object 6 in the virtual space 11. Here, the effect object 24 is, for example, a gift box. In the effect using the effect object 24, a countdown starts at the same time as the gift box is placed, and the lid is opened when the count value reaches 0. Furthermore, the effect ends when a predetermined time (2 seconds) has elapsed since the effect object 24 was placed in the virtual space 11.

[0227] Referring to Figure 19, an object of an item specified by an item insertion operation is placed in a virtual space 11 defined in each of multiple user terminals 800 by receiving action instruction data corresponding to the object from computer 200.

[0228] That is, in the user terminal 800 that performed the item insertion operation, the effect object 24 is placed in the virtual space 11 defined by that user terminal 800 in accordance with that item insertion operation, and the item object is placed based on the action instruction data transmitted when a predetermined time has passed since that placement. On the other hand, in the other user terminals 800, the effect object 24 is not placed in the virtual space 11, but the item object is placed in that virtual space 11 at the same timing as when the item object is placed in the user terminal 800 that performed the item insertion operation. When the processing of step S13 is completed, or if it is not determined in step S09 that an item insertion operation has been performed, the process proceeds to step S14.

[0229] The user 8 can also input and send a comment on the avatar object 6 by operating the user terminal 800. The comment is sent to the computer 200 during the viewing period (for example, the period during which the processing of steps S05 to S14 is executed). When the computer 200 receives the comment from the user terminal 800, it can display the user name on the display 430. The player 5 can input a comment in response to the comment. The comments input by the user and the player 5 are distributed from the computer 200 to multiple user terminals 800. In each user terminal 800, the comment is displayed on the touch screen 870, superimposed on the field of view image 1400.

[0230] In step S14, it is determined whether or not a viewing end operation has been performed by user 8 based on an operation input from touch screen 870. If it is determined that a viewing end operation has not been performed, the process returns to step S05. As a result, the process of controlling the movement, placement, etc. of the object is repeatedly executed every time action instruction data is received from computer 200.

[0231] On the other hand, if it is determined that a viewing end operation has been performed, viewing end information is transmitted to computer 200 in step S15, and then the process ends.

[0232] 21, in step S21, it is determined whether it is time to start a stage of the game based on game parameters indicating the progress of the game. If it is determined that it is time to start a stage, the process proceeds to step S22. In step S22, the current number of viewers, i.e., the number of user terminals 800 currently watching the game, is determined based on viewing start information transmitted from the user terminal 800 when viewing starts (step S02 in FIG. 20) and viewing end information transmitted from the user terminal 800 when viewing ends (step S15 in FIG. 20).

[0233] In step S23, a point adjustment amount for adjusting the points associated with the item inserted by user 8 in a positive (increasing) direction is determined according to the number of viewers. Specifically, the point adjustment amount is increased as the number of viewers increases. For example, when the number of viewers is 0, the point adjustment amount is set to a 0% increase; when the number of viewers is 1 or more but less than 100, the point adjustment amount is set to a 5% increase; when the number of viewers is 101 or more but less than 200, the point adjustment amount is set to a 10% increase; and when the number of viewers is 201 or more, the point adjustment amount is set to a 20% increase.

[0234] As will be described later, the points are awarded to the avatar object 6. When the awarded points reach a predetermined value, the avatar object 6 can acquire an item by performing a gacha operation. By using the acquired item, the avatar object 6 can increase the possibility of achieving the achievement condition, thereby making the game more advantageous for the player.

[0235] In step S24, the difficulty level of the stage about to start and the degree of the support effect for supporting the avatar object 6 are adjusted according to the number of viewers watching the stage. Specifically, the stage difficulty is adjusted so that the achievement conditions become more difficult to achieve as the number of viewers increases, by increasing the combat ability and appearance number of enemy objects 23 and decreasing the combat ability of the avatar object 6. Furthermore, the degree of the support effect is adjusted so that the support effect decreases as the number of viewers increases, by decreasing the amount of recovery of the remaining number of bullets in the gun or the amount of recovery of the avatar object 6's stamina, and by decreasing the size, strength, weight, etc. of obstacles.

[0236] For example, when the number of viewers is 0, the combat ability and appearance amount of the enemy object 23, the combat ability of the avatar object 6, the recovery amount of the remaining bullets in the gun, the recovery amount of the stamina of the avatar object 6, and the size, strength, and weight of the obstacles are set to their initial values. When the number of viewers is 1 or more but less than 100, the combat ability and appearance amount of the enemy object 23 are set to be 5% higher than their initial values, the combat ability of the avatar object 6 is set to be 5% lower than their initial values, and the recovery amount of the remaining bullets in the gun, the recovery amount of the stamina of the avatar object 6, and the size, strength, and weight of the obstacles are set to be 5% lower than their initial values.

[0237] When the number of viewers is 101 or more but less than 200, the combat ability and appearance amount of the enemy object 23 are set to be 10% higher than their initial values, the combat ability of the avatar object 6 is set to be 10% lower than their initial values, and the recovery amount of the remaining bullets in the gun, the recovery amount of the stamina of the avatar object 6, and the size, strength, and weight of the obstacles are set to be 10% lower than their initial values.When the number of viewers is 201 or more, the combat ability and appearance amount of the enemy object 23 are set to be 20% higher than their initial values, the combat ability of the avatar object 6 is set to be 20% lower than their initial values, and the recovery amount of the remaining bullets in the gun, the recovery amount of the stamina of the avatar object 6, and the size, strength, and weight of the obstacles are set to be 20% lower than their initial values.

[0238] The combat ability of the enemy object 23 is determined, for example, by the movement speed (agility) of the enemy object 23 and its stamina at the start of the battle in the current stage. The number of enemy objects 23 that appear is determined, for example, by the total number of enemy objects 23 that appear in the current stage and the frequency of appearance of the enemy objects 23 in the current stage. The combat ability of the avatar object 6 is determined, for example, by the movement speed (agility) of the avatar object 6, its stamina at the start of the battle in the current stage, and correction values ​​according to the types of weapons and armor equipped.

[0239] When the process of step S24 is completed, the process proceeds to step S25. If it is not determined in step S21 that the stage has started, the process proceeds to step S25 without executing the processes of steps S22 to S24.

[0240] In step S25, it is determined based on an input from the communication IF 250 whether or not item input information has been received from the user terminal 800. If it is determined that the item input information has been received, the process proceeds to step S26, where cheering points are awarded to the avatar object 6. Specifically, the amount of points included in the item input information is corrected based on the point adjustment amount specified in step S23, and the corrected amount of points is awarded to the avatar object 6. For example, if the amount of points included in the item input information is 50 points and the point adjustment amount specified in step S23 is a 10% increase, 55 points (= 50 × 1.1) are awarded to the avatar object 6 as cheering points.

[0241] In step S27, if the item insertion information includes information for specifying the item insertion range, any position within the item insertion range is specified as the item insertion position. However, the center position of the item insertion range may be specified as the item insertion position, or a position within the item insertion range that corresponds to the positional relationship between the avatar object 6 and the enemy object 23 (for example, a position between the avatar object 6 and the enemy object 23 that is closest to the avatar object 6) may be specified as the item insertion position.

[0242] When the processing of step S27 is completed, the process proceeds to step S28. If the item insertion information does not include information for specifying the item insertion range, the process proceeds to step S28 without executing the processing of step S27. In step S28, based on the item insertion information, the type of item specified by the item insertion operation and the timing at which the item object is placed in the virtual space 11 to activate the support effect are identified. Specifically, the type of item and the activation timing of the support effect (i.e., the timing at which the predetermined time has elapsed since the item insertion information was received) are set in an activation management table (not shown).

[0243] When the processing of step S28 is completed, the process proceeds to step S29. Also, when it is determined in step S25 that the item insertion information has not been received, the process proceeds to step S29. In step S29, it is determined based on a timer (not shown) provided in the computer 200 whether or not the activation timing (item object placement timing) set in the activation management table has arrived.

[0244] If the item insertion information has not yet been received, or if the item insertion information has been received but the timing for the item insertion has not yet arrived, it is not determined in step S29 that the timing for the item insertion has arrived. In this case, the process proceeds to step S31 without executing the process of step S30.

[0245] On the other hand, if it is determined that the activation timing has arrived, the process proceeds to step S30, where the support effect corresponding to the type of item set in the activation management table is activated. At this time, the user ID of the user who inserted the item may be distributed to multiple user terminals 800. In each user terminal 800, an image representing the user ID is displayed on the touch screen 870, superimposed on the field of view image 1400.

[0246] Specifically, in step S30, if the item is a magazine, the number of remaining bullets in the gun used by the avatar object 6 is restored, thereby giving the player 5 an advantage in the game progress; if the item is a first aid kit, the avatar object 6's stamina is restored, thereby giving the player 5 an advantage in the game progress. If the item is an obstacle, the obstacle object is placed at the item insertion position identified in step S27 among positions in the virtual space 11, thereby giving the player 5 an advantage in the game progress. The number of bullets, the amount of stamina restored, and the size, strength, weight, etc. of the obstacle are determined according to the degree of the support effect adjusted in step S24. When the processing of step S30 is completed, the process proceeds to step S31.

[0247] In step S31, it is determined whether or not a gacha operation has been performed by the avatar object 6 based on an input from the controller 300 or the motion sensor 420. The gacha operation is enabled when the cheering points given to the avatar object 6 reach a predetermined value. If it is determined that the gacha operation has been performed, the process proceeds to step S32, where the cheering points given to the avatar object 6 are consumed to obtain an item.

[0248] The items acquired by the gacha operation include, for example, unique items that are different from the items that can be inserted by the user 8. Furthermore, the items acquired by the gacha operation include items that can be used regardless of whether or not the user has provided support. Note that the acquired items may be the same items as the items that can be inserted by the user 8, or may be items that can be used with support from the user 8.

[0249] In step S33, a fulfillment possibility improvement effect (an effect that improves the possibility of fulfilling the achievement condition) corresponding to the acquired item is activated. Specifically, an object of the item (e.g., a knife) is placed on the floor in front of the avatar object 6 (e.g., dropping the knife). The avatar object 6 can use the object to fight against the enemy object 23. This increases the possibility of clearing the game. Note that the power (e.g., size) of the object may be determined according to the degree of the support effect adjusted in step S24.

[0250] When the processing of step S33 is completed, the process proceeds to step S34. If it is not determined in step S31 that a gacha operation has been performed, the process proceeds to step S34 without executing the processing of steps S32 and S33. In step S34, the game is progressed based on the game program. Specifically, the avatar object 6, enemy object 23, and other objects (including the item object inserted in step S30) are updated in the virtual space 11 by being positioned, moved, changed, etc., in a manner corresponding to the difficulty level adjusted in step S24. Furthermore, various game parameters such as the stamina of the avatar object 6, the number of available magazines, the number of remaining bullets in the gun, and the number of remaining enemy objects 23 are initialized or updated in a manner corresponding to the difficulty level.

[0251] In step S35, action instruction data is generated based on the objects in the virtual space 11 updated in step S34, to enable identification of the position, action, etc. of the avatar object 6 and the positions, actions, etc. of other objects such as the enemy object 23, and the action instruction data is distributed to multiple user terminals 800. If an obstacle object is placed in the virtual space 11 by the processing of step S30, action instruction data for identifying that the object has been placed is distributed in this step. In step S35, game information is generated including various game parameters that were initially set or updated in step S34, and the game information is distributed to multiple user terminals 800.

[0252] As described above, the process according to the flowchart shown in Fig. 21 is executed at predetermined time intervals (1 / 60 seconds). Therefore, the action instruction data and game information are distributed to a plurality of user terminals 800 via the server 600 at the predetermined time intervals. At each user terminal 800, game play video is displayed on the touch screen 870 based on the action instruction data and game information. When the process of step S35 is completed, the process returns.

[0253] <Effects of this embodiment> According to this embodiment, gameplay footage of a game that is cleared when an avatar object 6 appearing in virtual space 11 fulfills a predetermined achievement condition is displayed on each of a plurality of user terminals 800 based on action instruction data distributed from computer 200. Computer 200 identifies the number of user terminals 800 displaying the gameplay footage, i.e., the number of viewers, and adjusts the difficulty level for fulfilling the achievement condition according to the number of viewers.

[0254] This encourages viewers to support the game by inserting items, etc. In other words, the greater the number of viewers, the higher the difficulty level, and therefore, it is expected that an increase in the amount of virtual currency consumed will occur as many viewers insert items.

[0255] Furthermore, according to this embodiment, when the computer 200 receives item insertion information from the user terminal 800 in response to an item insertion operation by the user 8, the computer 200 activates a support effect that supports the achievement of the achievement condition. This allows the user 8 to actively participate in the gameplay by the player 5, thereby increasing the user's interest in the game and their desire to watch it.

[0256] Furthermore, according to this embodiment, the item insertion operation is an operation that involves consuming virtual currency owned by the user 8. As a result, the user 8 will carefully perform the item insertion operation while assessing the progress of the game by the avatar object 6 and the support provided by other users 8, which can make the game more interesting than if an unlimited number of items were allowed to be inserted.

[0257] Furthermore, according to this embodiment, the degree of the support effect that supports the achievement of the achievement condition is adjusted according to the number of viewers. Specifically, the greater the number of viewers, the smaller the degree of the support effect. As a result, when the number of viewers is large, it is expected that there will be more opportunities for support from viewers, but the degree of the support effect will decrease as the difficulty level increases. On the other hand, when the number of viewers is small, it is expected that there will be fewer opportunities for support from viewers, but the degree of the support effect will increase as the difficulty level decreases. As a result, it is expected that the possibility that player 5 will be able to complete the game will converge within a certain range regardless of the number of viewers, and it is possible to prevent the possibility of player 5 completing the game from changing drastically (becoming easier or more difficult) depending on the number of viewers.

[0258] Furthermore, according to this embodiment, points are associated with inserted items, and the points are granted to the avatar object 6 by an item insertion operation. When the granted points reach a predetermined value, the avatar object 6 can insert items into the virtual space 11 by a gacha operation. The player 5's chances of clearing the game are improved by inserting items by the gacha operation. Furthermore, the amount of positive adjustment of the points granted to the avatar object 6 varies depending on the number of viewers. Specifically, the amount of positive adjustment increases as the number of viewers increases. This can motivate the player 5 to distribute higher-level gameplay videos or gameplay videos that are more entertaining to viewers, in order to increase the number of viewers and enable the player 5 to earn more points by inserting items.

[0259] Furthermore, according to this embodiment, the amount of points associated with an item is set to an amount corresponding to the amount of virtual currency consumed when the item is inserted, which allows the feelings of the user 8 (the extent of their willingness to support) to be reflected in the amount of points granted to the avatar object 6.

[0260] Furthermore, according to this embodiment, the game includes multiple stages, each of which is associated with a completion condition, and the difficulty level of each stage is adjusted according to the number of viewers at the start of that stage. This allows for more detailed difficulty adjustment than, for example, a system that adjusts the difficulty level according to the number of viewers at the start of the first stage. Furthermore, the processing burden for adjusting the difficulty level can be reduced compared to a system that adjusts the difficulty level by constantly identifying the number of viewers even during a stage.

[0261] Furthermore, according to this embodiment, the game is a fighting game in which the avatar object 6 fights against enemy objects 23, and the fighting ability of the avatar object 6 and the fighting ability and appearance number of the enemy objects 23 are adjusted according to the number of viewers. Specifically, the fewer the number of viewers, the higher the fighting ability of the avatar object 6, and the more viewers there are, the higher the fighting ability of the enemy objects 23 and the more enemy objects 23 appear. This makes it less likely that a situation will arise in which the player 5 has difficulty clearing the game due to a small number of viewers. It also makes it possible to properly motivate viewers to support the player 5.

[0262] Furthermore, according to this embodiment, the game progresses in a manner according to the difficulty level, and game information including game parameters that indicate the progress of the game is distributed to the user terminal 800. This makes it possible to display on the user terminal 800 the strength of the avatar object 6, the number of available magazines, the number of remaining bullets in the gun, the number of remaining enemy objects 23, etc.

[0263] Furthermore, according to this embodiment, the difficulty level is adjusted according to the number of viewers, so that the player 5 operating the avatar object 6 is motivated to improve his or her game operation skills and distribute more advanced gameplay footage or gameplay footage that is entertaining to viewers, thereby further increasing the number of viewers and distributing even more difficult and advanced gameplay footage. As a result, a virtuous cycle is promoted: improved game operation skills → increased number of viewers → improved game operation skills → ... Furthermore, adjusting the difficulty level makes it difficult for viewers to predict the development of the game, thereby increasing the interest in gameplay footage and the viewers' willingness to watch.

[0264] According to this embodiment, the user terminal 800 displays gameplay video representing the virtual space 11 of the game on the touch screen 870. Furthermore, upon receiving an item insertion operation from the user 8 for specifying an item object to be placed in the virtual space 11, the user terminal 800 of the user 8 transmits item insertion information to the computer 200 via the server 600, and displays an effect image using the effect object 24 on the touch screen 870.

[0265] The computer 200 places the item in the virtual space at a timing according to the item insertion information, generates action instruction data for the item, and distributes the action instruction data to multiple user terminals 800 via the server 600. The user terminal 800, while watching the game, places the item object specified by the user in the virtual space 11 based on the action instruction data. Furthermore, the amount of action instruction data is significantly smaller than, for example, object data, and the time required to generate and transmit the action instruction data can be shortened. This allows the placement of item objects to be synchronized among multiple user terminals 800, and also minimizes the time from the item insertion operation to the placement of the item object.

[0266] Furthermore, in the virtual space 11 defined by the user terminal 800 where the item insertion operation is performed, a representation image using the representation object 24 is displayed until the item object is placed. This allows for a period of time from when the user performs the item insertion operation until the item object is placed in the virtual space 11, and also prevents the user from feeling uncomfortable.

[0267] Furthermore, according to this embodiment, the effect image is displayed by arranging the effect object 24 in the virtual space 11. Therefore, even when an image other than a virtual object (for example, an image showing the user ID of the user 8 who has inserted an item, or a comment entered by the user or player) is superimposed on the field of view image 1400, it is possible to prevent the image other than the superimposed virtual object from being hidden by the effect object 24 and becoming impossible to see, and it is possible to prevent the appeal of the gameplay video from being reduced.

[0268] Furthermore, according to this embodiment, while multiple types of item objects are provided, the effect object 24 is defined as an object that is independent of the type of inserted item object, thereby minimizing the processing load on the user terminal 800.

[0269] Furthermore, according to this embodiment, the display data for displaying an object in the virtual space 11 is not received at the timing of placing the object in the virtual space 11, but is pre-stored in the storage 830. Each of the multiple user terminals 800 watching the game places the item object in the virtual space 11 using the display data based on the action instruction data. Furthermore, the user terminal 800 on which the item insertion operation has been performed executes an effect using the effect object 24 for a predetermined period of time.

[0270] Here, the item insertion information includes the timing at which the effect by the effect object 24 ends (the timing at which the support effect is activated), and when the user terminal 800 receives the item insertion operation, it transmits the item insertion information to the computer 200 via the server 600. As a result, the computer 200 places the item object in the virtual space 11 at the timing specified from the item insertion information, and generates action instruction data and transmits it to each of the multiple user terminals 800. As a result, the user terminal 800 that performed the item insertion operation can place the item object in the virtual space 11 at the same time as the effect by the effect object 24 ends. In other words, the effect by the effect object 24 and the placement of the item object can be seamlessly connected.

[0271] Furthermore, according to this embodiment, the user terminal 800 places the virtual camera 20 in the virtual space 11 and displays a field of view image 1400 representing the field of view of the virtual camera 20 on the touch screen 870. At this time, the user terminal 800 specifies an item throwing range according to the range of the field of view image 1400 and transmits item throwing information including the item throwing range to the computer 200 via the server 600. This allows an item object to be placed in the field of view of the virtual camera 20. Furthermore, the item throwing range is changed in accordance with the position and orientation of the virtual camera 20 by a user operation. This enables strategic support, such as throwing an obstacle object between the avatar object 6 and the enemy object 23.

[0272] Furthermore, according to this embodiment, the position at which the inserted item object is to be placed is determined by the computer 200 from within the item insertion range. At this time, the computer 200 can determine the position taking into consideration the positional relationship with other objects that have already been placed, and can distribute action instruction data for specifying the determined position. As a result, it is possible to prevent unexpected collisions or interference between the item object inserted by the operation of the user 8 and other objects that have already been placed.

[0273] <Modification> Modifications of the above-described embodiment are listed below.

[0274] (1) In the above embodiment, the number of points given to the avatar object 6 is changed depending on the number of viewers. However, instead of or in addition to changing the number of points, a predetermined value that the avatar object 6 must reach to perform a gacha operation may be changed depending on the number of viewers.

[0275] (2) In the above embodiment, the item insertion information includes the timing at which the effect produced by the effect object 24 ends, and the user terminal 800 transmits the item insertion information to the computer 200 via the server 600 when it receives an item insertion operation. However, the item insertion information may be transmitted to the computer 200 via the server 600 when the effect ends (when a predetermined time has elapsed) without including the timing at which the effect ends in the item insertion information. In this case, the computer 200 may place the corresponding item upon receiving the item insertion information and generate and distribute action instruction data. The amount of action instruction data is significantly smaller than the amount of object data. Therefore, even by transmitting the item insertion information when the effect ends, it is possible to place the item object in the virtual space 11 and transmit the action instruction data to each of the multiple user terminals 800 almost simultaneously with the end of the effect produced by the effect object 24. Furthermore, if the performance time of the performance object 24 is fixed and known on the computer 200 side, it is possible to place the item object in the virtual space 11 at the same time as the performance of the performance object 24 ends and send the action instruction data to each of the multiple user terminals 800, even if the timing of the end of the performance is not included in the item insertion information.

[0276] (3) In the above embodiment, when an item insertion operation is performed, virtual currency owned by user 8 is consumed. Here, the amount of virtual currency consumed may be varied depending on the number of viewers. Even when the same item is inserted, for example, the greater the number of viewers, the more virtual currency is required (the greater the amount consumed). In this case, the amount of points associated with the item may be set to a fixed value regardless of the amount of virtual currency consumed.

[0277] (4) In the above embodiment, when the difficulty level is lowered, both the combat ability and the number of enemy characters that appear are reduced. However, the difficulty level may be comprehensively evaluated based on the combat ability and the number of enemy characters that appear. As long as the difficulty level is lowered, the combat ability may be increased while the number of enemy characters that appear is reduced, or the combat ability may be decreased while the number of enemy characters that appear is increased. Furthermore, when adjusting the difficulty level, only one of the combat ability and the number of enemy characters that appear may be changed.

[0278] (5) In the above embodiment, the number of viewers of the current stage is not displayed on the gameplay video. However, the number of viewers may be displayed on the gameplay video. Alternatively, or in addition, information that can identify the difficulty level according to the number of viewers may be displayed on the gameplay video.

[0279] (6) In the above embodiment, the player 5 wears the HMD set 110 and controls the movement of the avatar object 6. However, the player 5 may display the avatar object 6 on a stationary monitor such as a monitor for a desktop PC and control the movement of the avatar object 6 using a hand controller or a keyboard. Alternatively, the player 5 may display the avatar object 6 on the screen of a mobile information terminal and control the movement of the avatar object 6 by touching the screen.

[0280] (7) In the above embodiment, the item object is thrown into the virtual space 11 in response to a tap operation on an icon on the UI image 22. However, the item object may be thrown into the virtual space 11 in response to a flick operation or a swipe operation. In this case, the item throwing range may be determined based on the manner of the flick operation (operation direction) or the manner of the swipe operation (speed, time or distance of the touch operation). Furthermore, the item object may be thrown into the virtual space 11 in response to a voice input instead of a touch operation on the screen.

[0281] (8) In the above embodiment, the user terminal 800 specifies an item insertion range, and the computer 200 specifies any position within the item insertion range as the item insertion position. However, the user terminal 800 may directly specify the position where the inserted item object is to be placed (item insertion position). In this case, if the item object is inserted into the virtual space 11 in response to a flick operation or a swipe operation, the item insertion position may be specified based on the manner of the flick operation or swipe operation (speed, time or distance of the touch operation). Furthermore, the item insertion position may be specified by selecting the item to be inserted by a touch operation, and then subsequently touching the position where the item is to be placed, or by moving the touch operation position where the item to be inserted was selected to the position where the item is to be placed and then releasing the touch operation.

[0282] (9) In the above embodiment, when a viewpoint change operation is performed, the viewpoint of virtual camera 20 changes. At this time, the viewpoint of virtual camera 20 may be set to the viewpoint of avatar object 6 or the viewpoint of enemy object 23 in response to a specific viewpoint change operation (such as a touch operation on a predetermined viewpoint change icon). Also, when the viewpoint is the viewpoint of enemy object 23, and the enemy object 23 is defeated, the viewpoint of virtual camera 20 may be placed in the initial position or set to the viewpoint of another enemy object 23.

[0283] (10) In the above embodiment, an example was described in which the greater the number of viewers, the greater the combat ability and appearance frequency of the enemy objects 23, the greater the combat ability of the avatar object 6, and the greater the support effect for supporting the avatar object 6. However, instead of this, the greater the number of viewers, the greater the combat ability and appearance frequency of the enemy objects 23, the greater the combat ability of the avatar object 6, and the greater the support effect for supporting the avatar object 6. In other words, the greater the number of viewers, the lower the difficulty level for the player 5 to achieve the game's achievement conditions, and the smaller the number of viewers, the higher the difficulty level.

[0284] (11) In the above embodiment, the combat ability of the avatar object 6 or the enemy object 23 is determined based on the stamina at the start of the battle in the current stage, and the stamina is increased or decreased depending on the number of viewers. However, instead of or in addition to increasing or decreasing the stamina, the amount of decrease in stamina due to damage received may be increased or decreased depending on the number of viewers.

[0285] (12) In the above embodiment, the enemy object 23 is a non-player object that operates according to a game program and cannot be directly controlled by the player 5. However, the enemy object 23 may be a player object that can be directly controlled by another player 5. In this case, the nature of the enemy object 23 may be different for each stage, such that a certain enemy object 23 becomes a player object in one stage and a non-player object in another stage. Furthermore, the enemy object 23 may be a player object that can be controlled by a certain user using a user terminal.

[0286] (13) In the above embodiment, the positive adjustment amount of the points given to the avatar object 6 increases as the number of viewers increases. However, the positive adjustment amount may also increase as the number of viewers decreases. In this case, the user 8 is motivated to actively support the unpopular avatar object 6 and have the player 5 remember the user 8.

[0287] (14) In the above embodiment, when an item insertion operation is performed on the user terminal 800, a predetermined presentation object 24, such as a gift box, regardless of the inserted item, is placed in the virtual space 11 on the user terminal 800. However, when an item insertion operation is performed, an object of the inserted item may be displayed in a predetermined position instead of the presentation object 24. In this case, when a predetermined time has passed since the item insertion operation, the presentation by the presentation object 24 ends, and an object based on the action instruction data delivered from the computer 200 is placed in the virtual space 11.

[0288] (15) In the above embodiment, it is assumed that gameplay footage of a game in which an avatar object 6 competes against an enemy object 23 is displayed on the user terminal 800. However, it is also possible to display an avatar object that simply talks on the user terminal 800.

[0289] (16) In the above embodiment, the timing to place an item object inserted by the user 8 in the virtual space 11 is determined based on the action instruction data distributed from the computer 200 for both the user terminal 800 that performed the item insertion operation and the other user terminals 800. However, the placement timing for the user terminal 800 that performed the item insertion operation can be determined without being based on the action instruction data. Therefore, the item object may be placed in the virtual space 11 by executing different processes for the user terminal 800 that performed the item insertion operation and the other user terminals 800. Specifically, the user terminal 800 that performed the item insertion operation may perform a process to place the item object a predetermined time after the item insertion operation, while the other user terminals 800 may perform a process to place the item object based on the action instruction data distributed from the computer 200.

[0290] (17) In the above embodiment, multiple stages are prepared, and the difficulty level of the game is adjusted according to the number of viewers at the start of each stage. However, the number of viewers may be grasped in real time (or at regular intervals (1 minute)) and the difficulty level may be adjusted even during a stage.

[0291] (18) In the above embodiment, it is assumed that the operation of the user 8 (such as an item insertion operation) assists the avatar object 6 in achieving the achievement conditions in the game that is being progressed. However, the operation of the user 8 may have an obstructing effect that hinders the achievement conditions in the game that is being progressed by the avatar object 6 (puts the game at a disadvantage to the player 5). Specifically, the operation of the user 8 may provide a first aid kit to the enemy object 23 in the above embodiment to restore a game parameter related to the strength of the enemy object 23, or may throw an axe or hoe that the enemy object 23 can use into the virtual space 11 to increase a game parameter related to the attack power of the enemy object 23, thereby obstructing the game progress of the avatar object 6 (i.e., putting the game progress at a disadvantage to the player 5).

[0292] (19) In the above embodiment, the number of viewers is classified in units of 100, and the greater the number of viewers in the range of 200 or less, the greater the increase in the difficulty level. However, the increase in the difficulty level does not necessarily have to correspond to the increase in the number of viewers. That is, the increase in the difficulty level when the number of viewers is 1 or more but less than 100 may be greater than the increase in the difficulty level when the number of viewers is 101 or more but less than 200. Furthermore, the difficulty level is not limited to being adjusted in units of 100 viewers, but may be adjusted in units of 1 viewer, or may be adjusted in larger units as the number of viewers increases (0 to 10, 11 to 50, 51 to 150, 151 to 300, etc.).

[0293] (20) In the above embodiment, a fighting game is assumed in which the avatar object 6 fights against the enemy object 23, and the difficulty level is determined based on the combat ability and appearance number of the enemy object 23 and the combat ability of the avatar object 6. In addition, items such as magazines, first aid kits, and obstacles are provided as items that the user 8 can insert to support the avatar object 6.

[0294] However, if the game is envisioned as a puzzle game in which the avatar object 6 rearranges the placement of blocks, the difficulty level may be determined based on the number of blocks, size of the blocks, type of block, etc. required to achieve the achievement conditions, and items that the user 8 can throw may include hammers or bombs to destroy objects in the way (unnecessary blocks, walls, ornaments).

[0295] Furthermore, assuming that the game is an escape game in which the avatar object 6 must escape from a locked room, the difficulty level may be determined based on the variety of objects placed in the room, the location of the key for escape, the number of escape support items that suggest the location of the key, etc., and any of the escape support items may be provided as an item that the user 8 can insert. In this case, the degree of support effect is determined based on the number of escape support items that can be inserted.

[0296] (21) In the above embodiment, an example was described in which the effect object 24 is positioned so that the effect image is displayed in a position that does not overlap with the avatar object 6 and does not obstruct the visibility of the avatar object 6. However, the effect object 24 may be positioned so that it is displayed in a position that partially overlaps with the avatar object 6, or may be a transparent object (e.g., a translucent object).

[0297] (22) In the above embodiment, the effect image is displayed by placing the effect object 24 in the virtual space 11. However, instead of placing the effect object 24 in the virtual space 11, the effect image may be displayed on the touch screen 870 by superimposing the effect image on the field of view image 1400.

[0298] (23) In the above embodiment, the item insertion information includes information for identifying the item insertion range. However, instead of the information for identifying the item insertion range, the item insertion information may include information for identifying the viewpoint of virtual camera 20.

[0299] (24) In the above embodiment, when placing an item object in the virtual space 11 shown in Fig. 8, the computer 200 may generate action instruction data for specifying the placement and placement timing before placing the item object, and distribute the generated action instruction data to the user terminal 800. In this case, the action instruction data includes information for specifying the placement and action of the item object, as well as information for specifying the timing to place the item object. Furthermore, the user terminal 800 places the object in the virtual space 11 shown in Fig. 14 at the specified timing.

[0300] (25) In the above embodiment, the computer 200 distributes the action instruction data to the plurality of user terminals 800 at predetermined time intervals. However, the action instruction data may be distributed when a change occurs in an object in the virtual space 11 shown in FIG. 8.

[0301] (26) In the above embodiment, when cheering points are awarded to the avatar object 6, the amount of points included in the item insertion information is corrected based on the point adjustment amount identified in step S23. However, the point adjustment amount identified in step S23 may be included in the game information and delivered to the user terminal 800, and the point adjustment amount (or the adjusted points) may be displayed superimposed on the field of view image 1400 according to the item. In this case, the user can know how much the points awarded to the avatar object 6 will be adjusted (or the adjusted points) before performing the item insertion operation.

[0302] (27) In the above embodiment, the difficulty level of the game is adjusted by the computer 200. However, the difficulty level may be included in the game information and delivered to the user terminal 800, and may be displayed superimposed on the field of view image 1400, or the types and number of items that the user 8 can insert may be varied depending on the difficulty level.

[0303] (28) In the above embodiment, the game includes multiple stages, and the difficulty level is adjusted according to the number of viewers at the start of each stage. However, the game may include only one stage, and the difficulty level may be adjusted according to the number of viewers at the start of that stage. Furthermore, regardless of the number of stages, the difficulty level may be adjusted according to the number of viewers at a predetermined point in time during the game.

[0304] <Additional Notes> The matters described in the above embodiments will be supplemented below.

[0305] (Appendix 1): According to one aspect of an embodiment of the present disclosure, there is provided a distribution program executed by a computer (200) that includes a processor and memory and distributes data for enabling gameplay footage of a game that is cleared by fulfilling a predetermined condition to be displayed on a viewing terminal (800), wherein the distribution program causes the processor to execute a step (S22) of identifying the number of viewers on the viewing terminal displaying the gameplay footage, and a step (S24) of adjusting the difficulty level for fulfilling the predetermined condition in accordance with the number of viewers.

[0306] (Appendix 2): In (Appendix 1), the distribution program causes the processor to execute a step (S30) of exerting a support action to assist in the fulfillment of the specified condition in response to receiving operation information transmitted from the viewing terminal based on a specified operation by the viewer.

[0307] (Appendix 3): In (Supplementary Note 2), the predetermined operation is an operation that involves consuming virtual currency associated with the viewer.

[0308] (Appendix 4): In (Supplementary Note 2) or (Supplementary Note 3), the distribution program causes the processor to execute a step (S24) of changing the magnitude of the support action depending on the number of viewers.

[0309] (Appendix 5): In (Supplementary Note 4), the changing step changes the support action so that when the number of viewers is a first number, the support action is smaller than when the number of viewers is a second number smaller than the first number.

[0310] (Appendix 6): In any of (Appendix 2) to (Appendix 5), the game is a game that causes an item associated with points to appear by receiving the operation information, the exerting step exerts a support effect according to the item, and the distribution program causes the processor to execute a step (S26) of granting points associated with the item to the distributor, and a step (S33) of exerting an improving effect that increases the likelihood that the specified condition will be met when the points granted to the distributor reach a specified value.

[0311] (Appendix 7): In (Supplementary Note 6), the distribution program causes the processor to execute a step of changing the predetermined value in accordance with the number of viewers.

[0312] (Appendix 8): In (Supplementary Note 6) and (Supplementary Note 7), the distribution program causes the processor to execute a step (S23) of changing the number of points to be given to the distributor according to the number of viewers.

[0313] (Appendix 9): In any of (Appendix 6) to (Appendix 8), the specified operation is an operation that involves the consumption of virtual currency associated with the viewer, and the amount of points associated with the item is set according to the amount of virtual currency consumed to make the item appear.

[0314] (Appendix 10): In any one of (Supplementary Note 1) to (Supplementary Note 9), the adjusting step adjusts the difficulty level in accordance with the number of viewers when the game is started.

[0315] (Appendix 11): In (Appendix 10), the game includes a plurality of stages, each of which is associated with a predetermined condition, and the adjusting step adjusts the difficulty level for achieving the predetermined condition associated with each of the plurality of stages according to the number of viewers at the time each of the stages starts.

[0316] (Appendix 12): In any of (Appendix 1) to (Appendix 11), the data includes at least motion data indicating the movements of a broadcaster, the game is a game in which a player object that moves based on the motion data fights other objects, and the adjusting step adjusts the difficulty level by changing at least one of the ability of the player object, the ability of the other objects, and the appearance amount of the other objects, depending on the number of viewers.

[0317] (Appendix 13) In any of (Appendix 1) to (Appendix 12), the distribution program further causes the processor to execute the steps of: controlling the progress of the game in a manner corresponding to the difficulty level adjusted by the adjusting step (S34); generating object information for identifying the type and position of game objects to be placed in accordance with the progress of the game (S35); and generating parameter information for identifying game parameters to be updated in accordance with the progress of the game (S35), and the data includes at least the object information and the parameter information.

[0318] (Appendix 14): In (Appendix 1), the distribution program causes the processor to execute, when receiving operation information transmitted from the viewing terminal in response to a predetermined operation by a viewer, a step of exerting an inhibitory effect that prevents the specified condition from being met, and a step of changing the magnitude of the inhibitory effect in response to the number of viewers.

[0319] (Appendix 15): In (Supplementary Note 14), the data includes at least motion data indicating the movement of a broadcaster, the game is a game in which a player object that moves based on the motion data fights other objects, and the influencing step updates game parameters to put the player object at a disadvantage.

[0320] (Appendix 16): According to one aspect of an embodiment of the present disclosure, there is provided a distribution method executed by a computer (200) having a processor and memory, which distributes data for enabling gameplay footage of a game that is cleared by fulfilling a predetermined condition to be displayed on a viewing terminal (800), the distribution method including a step (S22) in which the computer identifies the number of viewers of the viewing terminals displaying the gameplay footage, and a step (S24) in which the computer adjusts the difficulty level for fulfilling the predetermined condition in accordance with the number of viewers.

[0321] (Appendix 17): According to one aspect of an embodiment of the present disclosure, a computer (200) distributes data for enabling gameplay footage of a game that is cleared by fulfilling a predetermined condition to be displayed on a viewing terminal (800), the computer (200) comprising: a memory unit (2200) that stores a distribution program; and a control unit (2100) that controls operation of the computer by executing the distribution program, wherein the control unit executes a step (S22) of identifying the number of viewers of the viewing terminal displaying the gameplay footage; and a step (S24) of adjusting the difficulty level for fulfilling the predetermined condition in accordance with the number of viewers.

[0322] (Appendix 18): According to one aspect of an embodiment of the present disclosure, there is provided a viewing program executed by a viewing terminal (800) having a processor, a memory, and a display unit, wherein the viewing program causes the processor to perform the following steps: sending information to a computer that adjusts the difficulty level for achieving a predetermined condition for clearing a game in accordance with the number of viewers of the viewing terminal displaying gameplay footage of the game, to enable the number of viewers to be identified (S02, S15); receiving data from the computer that enables the gameplay footage of the game with the adjusted difficulty level to be displayed on the viewing terminal (S05); and displaying the gameplay footage of the game with the adjusted difficulty level based on the data (S06-S08).

[0323] [Software implementation example] Each control block of the control units 2100, 6100, and 8100 may be realized by a logic circuit (hardware) formed on an integrated circuit (IC chip) or the like, or may be realized by software using a CPU (Central Processing Unit).

[0324] In the latter case, an information processing device including at least one of the control units 2100, 6100, and 8100 includes a CPU that executes instructions of a program, which is software that realizes each function; a ROM (Read Only Memory) or storage device (these are referred to as "recording media") in which the program and various data are recorded so as to be readable by a computer (or CPU); and a RAM (Random Access Memory) in which the program is deployed. The object of the present invention is achieved when the computer (or CPU) reads and executes the program from the recording media. The recording media can be "non-transitory tangible media," such as tapes, disks, cards, semiconductor memories, and programmable logic circuits. The program may also be supplied to the computer via any transmission medium capable of transmitting the program (such as a communication network or broadcast waves). Note that one aspect of the present invention can also be realized when the program is embodied by electronic transmission in the form of a data signal embedded in a carrier wave.

[0325] The present invention is not limited to the above-described embodiments, and various modifications are possible within the scope of the claims. Embodiments obtained by appropriately combining the technical means disclosed in different embodiments are also included in the technical scope of the present invention.

[0326] The embodiments disclosed herein should be considered to be illustrative in all respects and not restrictive. The scope of the present invention is defined by the claims, not by the above description, and is intended to include all modifications within the meaning and scope of the claims. [Explanation of symbols]

[0327] 2 Network, 5 Player, 6 Avatar object, 8 User, 11 Virtual space, 12 Center, 13 Panoramic image, 14, 20 Virtual camera, 15, 21 Field of view area, 16 Reference line of sight, 17, 1400 Field of view image, 18, 19 Area, 100 HMD system, 110 HMD set, 120 HMD, 130 Monitor, 140 Gaze sensor, 150 First camera, 160 Second camera, 170 Microphone, 180 Speaker, 190 Sensor, 200 Computer, 210, 610, 810 Processor, 220, 620, 820 Memory, 230, 630, 830 Storage, 240, 640, 840 Input / output interface, 250, 650, 850 Communication interface, 260, 660 Bus, 300 Controller, 300R Right controller, 300L Left controller, 310 Grip, 320 Frame, 330 Top panel, 340, 350, 370, 380 Buttons, 360 Infrared LED, 390 Analog stick, 410 HMD sensor, 420 Motion sensor, 430 Display, 510 Control module, 520 Rendering module, 530 Memory module, 540 Communication control module, 600 Server, 700 External device, 1421, 8121 Virtual object generation module, 1422, 8122 Virtual camera control module, 1423 Operation object control module, 1424, 8124 Avatar object control module, 1425 Motion detection module, 1426 Collision detection module, 1427, 8127 Virtual object control module, 800 User terminal, 860 Camera, 870 Touch screen, 880 Distance measurement sensor, 2100, 6100, 8100 control unit, 2200, 6200, 8200 memory unit, 2101, 8101 operation reception unit, 2102, 8102 display control unit, 2103, 8103 UI control unit, 2104, 8104 animation generation unit, 2105, 8105 game progression unit, 2106, 8106 virtual space control unit, 2107 reaction processing unit, 8107 progress information generation unit, 6101 progress support unit, 6102 sharing support unit, 231, 631, 831 game program, 132 game information, 133 user information, 1020 controller, 1030 storage medium

Claims

1. A distribution program executed by a computer that distributes data for enabling a viewing terminal to display gameplay footage of a game that is cleared by fulfilling a predetermined condition in a predetermined virtual space, The computer and in response to receiving operation information transmitted from the viewing terminal based on a predetermined operation by a viewer viewing the gameplay video, the device functions as a means for exerting a support action for supporting the establishment of the predetermined condition; the support action includes placing a specific object in the predetermined virtual space; the means for exerting an assisting effect is capable of adjusting the difficulty of achieving the predetermined condition by placing the specific object at a position in the predetermined virtual space that assists in achieving the predetermined condition, The location where the satisfaction of the predetermined condition is supported is any location within an area where the satisfaction of the predetermined condition can be supported.

2. The specific object is provided in a plurality of types, the operation information includes information for identifying the type of the specific object selected by the viewer; the means for exerting the support action places, as the support action, a specific object of a type identified from the operation information in the predetermined virtual space; 2. The distribution program according to claim 1, wherein the types and number of specific objects that the viewer can place in the predetermined virtual space vary depending on the degree of difficulty in achieving the predetermined condition.

3. The data includes at least motion data indicating the movement of the distributor, the game is a game in which a player object that moves based on the motion data fights against other objects, The computer and functioning as a means for identifying the number of viewers of the viewing terminals displaying the gameplay video; 3. The distribution program according to claim 1, wherein the means for exerting a support effect is capable of adjusting the difficulty level by changing at least one of the ability of the player object, the ability of the other objects, and the appearance amount of the other objects, depending on the number of viewers.

4. A system including: a distribution device that distributes data for displaying gameplay video of a game that is cleared by fulfilling a predetermined condition in a predetermined virtual space; and a viewing terminal that displays the gameplay video based on the data distributed from the distribution device, the viewing terminal transmits operation information based on a predetermined operation by a viewer viewing the gameplay video; The distribution device a means for providing a support action to support the establishment of the predetermined condition in response to receiving operation information transmitted from the viewing terminal; the support action includes placing a specific object in the predetermined virtual space; the means for exerting an assisting effect is capable of adjusting the difficulty of achieving the predetermined condition by placing the specific object at a position in the predetermined virtual space that assists in achieving the predetermined condition, The system, wherein the location where the satisfaction of the predetermined condition is supported is any location within an area where the satisfaction of the predetermined condition can be supported.

Images