Information processing system, program, and information processing method

Abstract

To make a plurality of users sharing a virtual space perform an interaction via an object display request smoothly.SOLUTION: An information processing system 10 for displaying an image in a virtual space to be viewed by a distribution user 13A and a viewing user 13B comprises: a setting unit that changes at least one of a position and direction of a virtual camera in the three-dimensional virtual space; a communication unit that acquires identification information, which is information transmitted from a user device 12 used by the viewing user 13B, for identifying an object selected by the viewing user 13B and a display request for the object; and an output control unit that on the basis of the identification information and display request for the object, sets a display position of the object depending on the position of the virtual camera and a position of a character object corresponding to the distribution user 13A and displays, on the display device, a virtual space image on which the object is arranged at the display position.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention relates to an information processing system, a program, and an information processing method. [Background technology]

[0002] For example, some videos featuring avatars display virtual space images rendered from objects included in the field of view of a virtual camera on a display. It is known that in these videos, a viewing user who watches the video provides an object as a gift to a broadcasting user who broadcasts the video (see, for example, Patent Document 1). A user device used by the viewing user transmits a display request for the object to be provided to a server that manages the broadcasting of the video. The user device of the broadcasting user and the user device of the viewing user display the object corresponding to the display request. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Patent Publication No. 2021-158558 Summary of the Invention [Problem to be solved by the invention]

[0004] Incidentally, some of the above-mentioned videos change the position and orientation of the virtual camera based on input operations by the user, thereby allowing the user to view any position in the virtual space.

[0005] However, when at least one of the position and direction of the virtual camera is changed, the provided object may move out of the field of view of the virtual camera. In this case, the broadcasting user may not be able to notice the object. As a result, the broadcasting user is unable to communicate with the viewing user, even though the viewing user provided the object. This problem can occur not only in video streaming, but also in any other application that displays a three-dimensional virtual space, such as a game. [Means for solving the problem]

[0006] An information processing system according to one embodiment of the present disclosure is an information processing system for displaying an image of a virtual space viewed by a first user and a second user, and includes: a setting unit that changes at least one of the position and direction of a virtual camera in the three-dimensional virtual space; a communication unit that acquires information transmitted from a user device used by the second user, the information including identification information for identifying an object selected by the second user and a display request for the object; and an output control unit that sets the display position of the object according to the position of the virtual camera and the position of a character object corresponding to the first user based on the identification information of the object and the display request, and displays a virtual space image in which the object is arranged at the display position on a display device.

[0007] A program according to one embodiment of the present disclosure causes one or more computers to function as a setting unit that changes at least one of the position and direction of a virtual camera in a three-dimensional virtual space, an output control unit that outputs a virtual space image, which is an image of the virtual space as seen from the virtual camera, to a display device viewed by a user, and a communication unit that acquires information transmitted from another device used by another user, the information including identification information for identifying an object selected by the other user and a display request for the object, and the output control unit sets the display position of the object according to the position of the virtual camera and the position of a character object corresponding to the user based on the identification information of the object and the display request, and outputs the object to the display position.

[0008] An information processing method according to one aspect of the present disclosure includes one or more computers changing at least one of the position and direction of a virtual camera in a three-dimensional virtual space, outputting a virtual space image, which is an image of the virtual space as seen from the virtual camera, to a display device viewed by a user, acquiring information transmitted from another device used by another user, which includes identification information for identifying an object selected by the other user and a display request for the object, setting a display position for the object based on the object identification information and the display request according to the position of the virtual camera and the position of a character object corresponding to the user, and outputting the object at the display position.

[0009] An information processing method according to one embodiment of the present disclosure is an information processing method using one or more computers to display an image of a virtual space viewed by a first user and a second user, wherein the computer changes at least one of the position and direction of a virtual camera in the three-dimensional virtual space, acquires information transmitted from a user device used by the second user, the information including identification information for identifying an object selected by the second user and a display request for the object, sets a display position of the object based on the object identification information and the display request according to the position of the virtual camera and the position of a character object corresponding to the first user, and displays a virtual space image with the object positioned at the display position on a display device. [Effects of the Invention]

[0010] According to the present disclosure, multiple users who share a virtual space can smoothly interact with each other through requests to display objects. [Brief explanation of the drawings]

[0011] [Figure 1] FIG. 1 is a schematic diagram illustrating an information processing system according to a first embodiment. [Figure 2]FIG. 2 is a diagram illustrating a hardware configuration of the information processing apparatus according to the embodiment. [Figure 3] FIG. 2 is a block diagram illustrating functions of a user device according to the embodiment. [Figure 4] FIG. 2 is a block diagram illustrating the functions of a management server according to the embodiment. [Figure 5] FIG. 2 is a diagram schematically illustrating user management information according to the embodiment. [Figure 6] FIG. 10 is a diagram schematically illustrating provided object information according to the embodiment. [Figure 7] FIG. 10 is a diagram schematically illustrating a display area of ​​a provision object according to the embodiment. [Figure 8] FIG. 10 is a diagram schematically illustrating a display area of ​​a provision object according to the embodiment. [Figure 9] 10 is a flowchart showing the procedure of a display process according to the embodiment. [Figure 10] FIG. 10 is a diagram illustrating an example of a provision object according to the embodiment. [Figure 11] FIG. 10 is a diagram illustrating collaboration distribution according to a second embodiment. [Figure 12] FIG. 11 is a diagram illustrating a display example of a provision object according to the third embodiment. [Figure 13] FIG. 10 is a diagram illustrating a display area of ​​a provision object in a modified example. DETAILED DESCRIPTION OF THE INVENTION

[0012] <Outline of the information processing system> An information processing system displays an image of a three-dimensional virtual space on a user device. The user device constituting the information system changes at least one of the position and direction of a virtual camera. The user device or server acquires information transmitted from a user device used by a second user, the information including identification information for identifying an object and a display request for the object. Based on the object identification information and the display request, the user device or server sets a display position for the object according to the position of the virtual camera and the position of a character object corresponding to the first user, and displays a virtual space image on the user device in which the object is arranged at the display position.

[0013] First Embodiment A first embodiment of an information processing system, a program, and an information processing method will be described below. In this embodiment, a distributor user distributes a video using a user terminal, and a viewing user views the video using a user terminal. The distributor user corresponds to a first user. The viewing user corresponds to a second user.

[0014] The video includes an image of a character object corresponding to the broadcasting user. The broadcasting user corresponds to the first user. The viewing user provides an object to the broadcasting user. This object is called a provided object. The viewing user is the second user. In this video distribution, the provided object is also called a gift object.

[0015] The provided object may be provided to support the broadcast user, etc. Furthermore, by providing the provided object, information about the user associated with the broadcast user may be changed.

[0016] <Information Processing System> An information processing system 10 will be described with reference to Fig. 1. The information processing system 10 includes a management server 11 and a user device 12. The management server 11 and the user device 12 are connected via a network NW.

[0017] In one aspect, the management server 11 is managed by a platform (PF) provider that provides a platform for distributing and viewing videos in a virtual space. The management server 11 comprises one or more devices. In other words, the management server 11 may be a group of servers. The management server 11 is connected to a user device 12 via a network NW such as the Internet. The user device 12 is a device on which a program for distributing and viewing videos is installed, or a device used by a user 13 who uses a video providing service for distributing and viewing videos using a web browser or the like. The management server 11 is responsible for relaying data related to video distribution.

[0018] FIG. 1 shows a broadcasting user 13A and a viewing user 13B. The broadcasting user 13A is a user 13 who broadcasts videos using a user device 12. The user 13B is a user 13 who watches the broadcasted videos on the user device 12. A single user 13 can be a broadcasting user 13A at one time and a viewing user 13B at another time. That is, the user 13 is a broadcasting user 13A when broadcasting videos, and a viewing user 13B when watching videos. The user device 12 that broadcasts videos is referred to as a broadcasting user device 12A, and the user device 12 that displays videos for viewing is referred to as a viewing user device 12B. The user device 12 can switch between a mode for broadcasting videos and a mode for displaying videos for viewing. In this embodiment, when the broadcasting user 13A and the viewing user 13B are described without distinction, they will simply be referred to as users 13. When the broadcasting user device 12A and the viewing user device 12B are described without distinction, they will simply be referred to as user devices 12.

[0019] <Hardware> The hardware of the user device 12 will be described with reference to FIG. 2. The user device 12 may be a smartphone (multifunction phone), a tablet device, a personal computer, a console game machine, a wearable computer, or any other information processing device capable of playing videos. The wearable computer may be, for example, a head-mounted display worn on the user's head or a glasses-type wearable device, as long as it has a screen on which the user can watch videos. The user device 12 may be a standalone device, or a set of multiple devices connected so that various data can be transmitted and received between them. An example of the latter user device 12 is a system that performs outside-in tracking.

[0020] The user device 12 includes a processing circuit 100, a storage 104, a sensor unit 112, a display 109, an input device 110, and peripheral devices 111. The user device 12 illustrated in Fig. 2 includes the sensor unit 112 that detects the movement of the broadcast user 13A. Note that the sensor unit 112 may be a separate device from the user device 12 that is communicatively connected to the user device 12.

[0021] The processing circuit 100 includes a CPU 101, which is a processor that executes one or more of the control processes disclosed in this specification, and a memory 102. Processing data and instructions may be stored in the memory 102. The processing data and instructions may also be stored in a storage 104, which is a storage medium disk such as a hard drive (HDD) or a portable storage medium, or in a storage medium provided separately from the processing circuit 100. The storage 104 is connected to the CPU 101 via a storage controller 103 and a bus 108. In this embodiment, an application is stored in the storage 104. The application includes a program for distributing and viewing videos and various data required to execute the program.

[0022] Each function disclosed herein may be implemented using circuitry, which may include, but is not limited to, a CPU 101, a general-purpose processor, a special-purpose processor, an integrated circuit, an ASIC (Application Specific Integrated Circuit), other conventional circuitry, and / or a combination thereof, configured or programmed to perform the disclosed functions. A processor is a processing circuit or circuit because it includes transistors and other circuits therein. A processor may be a programmed processor that executes a program stored in a memory. In this disclosure, a unit or means, such as a processing circuit, is hardware that performs or is programmed to perform the described functions. The hardware may be any hardware disclosed herein or otherwise known that is programmed or configured to perform the described functions.

[0023] Furthermore, the processor is not limited by the form of computer-readable medium on which the process instructions are stored. For example, the instructions may be stored on a CD, DVD, flash memory, RAM, ROM, PROM, EPROM, EEPROM, hard disk, or other non-transitory computer-readable medium of an information processing device with which processing circuit 100 communicates, such as a server or computer. The process may also be stored in network-based storage, cloud-based storage, or other mobile-accessible storage and executable by processing circuit 100.

[0024] Furthermore, the descriptions or blocks in the flowcharts disclosed herein can be understood as representing modules, segments, or portions of code that include one or more executable instructions for implementing specific logical functions or steps in a process. Furthermore, the descriptions or blocks in the flowcharts disclosed herein can perform two or more functions or steps in an order different from that shown or described in the specification, such as substantially simultaneously or in reverse order.

[0025] The hardware elements for implementing the processing circuit 100 may be implemented by various circuit elements. Furthermore, each function disclosed herein may be implemented by a circuit including one or more processing circuits.

[0026] The processing circuit 100 also includes a network controller 106 for connecting to a network NW. The network NW may be a public network such as the Internet, a private network such as a local area network (LAN) or a wide area network (WAN), or any combination thereof, and may also include a public switched telephone network (PSTN) or an integrated services digital network (ISDN®) or a sub-network. The network NW may also be a wired network such as an Ethernet® network or a universal serial bus (USB) cable, or a wireless network such as a cellular network including 3G, 4G, and 5G wireless cellular systems. The wireless network may also be Wi-Fi®, wireless LAN, Bluetooth®, or other known forms of wireless communication. Furthermore, the network controller 106 may be compliant with other direct communication standards such as Bluetooth®, near field communication (NFC), infrared, etc.

[0027] The processing circuit 100 further includes a display controller 105 and an input / output interface 107. The display controller 105, the network controller 106, and the input / output interface 107 are connected to a bus 108. The display controller 105 is connected to a display 109. The input / output interface 107 is connected to an input device 110 and peripheral devices 111.

[0028] The sensor unit 112 will now be described. The sensor unit 112 is one or more sensors that detect data indicating the movement of the user 13. The movement of the user 13 includes face motion, which indicates a change in the user's facial expression, and body motion, which indicates a change in the position and orientation of the user's body relative to the sensor unit 112. Face motion includes movements such as blinking and opening and closing of the mouth. A known sensor can be used as the sensor unit 112. The sensor unit 112 has a sensor that measures the distance to a measurement object by irradiating the measurement object with light. Examples of such sensors include true depth sensors or "LIDAR" (Light Detection and Ranging, or Laser Imaging Detection and Ranging) sensors. This allows the sensor unit 112 to capture accurate face data. The processing unit of the sensor unit 112 generates various information based on the depth map and the infrared image, compares the information with registered reference data, and calculates the depth of each point on the face (the distance between each point and the near-infrared camera) and positional deviations other than in the depth direction. In another example, the sensor unit 112 includes a ToF sensor that measures the time of flight (time of flight) between when light is irradiated onto a measurement object such as the user's face and when it is reflected back, or the phase difference between the irradiated light and the reflected light, using a distance measurement method other than the above-described methods. The ToF sensor calculates the distance to the measurement object based on the time of flight or phase difference. Alternatively or in addition to this, the sensor unit 112 may include a camera that captures an image of the user's face and an image processing unit that processes the data captured by the camera.

[0029] The sensor unit 112 outputs facial motion and body motion as tracking data to the processing circuit 100. The tracking data is data that detects the direction and position of parts of the user's body. The tracking data is included in the motion data. In this embodiment, a character object to which the motion data detected by the sensor unit 112 is applied is called an avatar object. The motion data refers to all data for moving an avatar object corresponding to the user 13.

[0030] For example, the motion data includes not only tracking data but also emote data. The emote data is data for making the avatar object perform a pre-registered predetermined movement, such as "clap." This pre-registered movement of the avatar object will be referred to as an "emote" hereinafter.

[0031] The sensor unit 112 may also have a function of tracking not only the user's face but also their hands (hand tracking). The sensor unit 112 may also include a sensor that detects the position or orientation of a human body part other than the hand. The sensor unit 112 may further include a sensor that detects speed or acceleration, a sensor that detects direction or orientation (such as a gyro sensor), etc. The sensor unit 112 may also have a spatial mapping function that recognizes objects in the real space where the user exists and maps the recognized objects onto a spatial map, that is, creates a map of the positions of the recognized objects in the real space.

[0032] The input device 110 may be a touch panel, a keyboard, a mouse, a controller operated by the user's hands (e.g., a game controller such as a game pad or joystick), or operation buttons provided on the housing of the user device 12. The controller may incorporate various known sensors, such as an acceleration sensor, a gyroscope, or an inertial measurement unit (IMU). In another example, the input device 110 may be a tracking device that identifies the user's hand movements, eye movements, head movements, gaze direction, and the like. In this embodiment, for example, the user's instructions can be determined based on the user's hand movements, and various operations can be performed, such as starting or ending video distribution, rating messages or videos, and displaying predetermined objects.

[0033] The peripheral device 111 includes a microphone that collects sounds in the real world, such as the voice of the user 13, and a speaker that outputs sound effects and the voices of other users 13, etc. The management server 11 has the same processing circuit 100 and storage 104 as the user device 12. In the management server 11, the input device 110, the peripheral device 111, the sensor unit 112, the display 109, and the like can be omitted.

[0034] <User device> The functions of the user device 12 will be described with reference to Fig. 3. The processing circuit 100 executes a program recorded in the storage 104, causing the user device 12 to function as a setting unit 15, a communication unit 16, and an output control unit 17. The user device 12 also includes a drawing information storage unit 18.

[0035] The setting unit 15 changes at least one of the position and the direction of the virtual camera in the three-dimensional virtual space. The communication unit 16 receives data for displaying a video transmitted by the broadcasting user device 12A (another device) from the management server 11. The communication unit 16 acquires information transmitted from another viewing user device 12B, including identification information for identifying the provided object selected by the viewing user 13B and a display request for the provided object.

[0036] The output control unit 17 displays a moving image, which is an image of the virtual space visually recognized by the user 13, on the display 109 based on data for displaying the moving image, and outputs the voice, etc., spoken by the broadcasting user 13A from the speaker. The output control unit 17 sets the display position of the provided object according to the position of the virtual camera and the position of the avatar object corresponding to the broadcasting user 13A, based on the identification information and display request of the provided object acquired by the communication unit 16. The output control unit 17 also displays the virtual space image in which the provided object is arranged at that display position on the display 109, which is a display device. A display area is associated with the display attribute.

[0037] The drawing information storage unit 18 stores drawing information. The drawing information includes data for displaying moving images, data for displaying setting screens, etc. The drawing information is updated by the management server 11 as appropriate.

[0038] <Administration Server> The management server 11 will be described with reference to Fig. 4. The management server 11 functions as a distribution management unit 20 and a user information management unit 21 by the processing circuit 100 thereof executing a program recorded in the storage 104. The management server 11 also includes a user management information storage unit 22 and a provided object information storage unit 23. These storage units 22 and 23 are configured by the storage 104.

[0039] The distribution management unit 20 starts distribution of the video in response to a distribution request from the distribution user device 12 A. Furthermore, the distribution management unit 20 transmits data for displaying the video to the viewer user device 12 B in response to a viewing request from the viewer user device 12 B.

[0040] The distribution management unit 20 receives, from the viewing user device 12B, identification information for identifying the provided object selected by the viewing user 13B and a display request for the provided object. The distribution management unit 20 also transmits the identification information for the provided object and the display request to the distribution user device 12A and the viewing user device 12B.

[0041] The user information management unit 21 updates the user management information 24 based on the compensation associated with the identification information of the provided object. If the provided object requires a compensation, the user information management unit 21 updates the user management information 24 of the viewing user 13B so as to subtract the compensation. In addition, the user information management unit 21 updates the user management information 24 of the distribution user 13A so as to add a reward for providing the provided object to the distribution user 13A.

[0042] FIG. 5 shows an example of the user management information 24 recorded in the user management information storage unit 22. The user management information 24 is recorded for each user. The user management information 24 includes a user ID (identifier), which is user identification information, and possession information. The possession information indicates value held by the user 13 for acquiring items and requesting the display of gift objects in an application or a platform that provides the application. The value includes, for example, coins. Coins can be used to acquire items, pay for lotteries (gacha), or pay for requesting the display of gift objects. Coins can be purchased by the user 13. The possession information may also include accumulated points awarded based on the user 13's activities. Activities include streaming and viewing videos, participating in events, etc. Points are awarded based on the user 13's activities, and the points are added to the accumulated points. The possession information may also include information regarding electronic money or cryptocurrency that can be used in the real world.

[0043] FIG. 6 shows an example of the provided object information 25 recorded in the provided object information storage unit 23. The provided object information 25 includes an object ID, an object name, a display attribute, consideration information, and remuneration information. The object ID is identification information of a provided object that can be selected by a viewing user. The object name is the name of the provided object. The display attribute indicates the attribute when the provided object is displayed. The display attribute may be a group or a type. The display attribute may also be a tag that can be assigned to an object. The consideration information is information indicating the consideration paid by the viewing user in order to provide the provided object. The consideration information is, for example, coins or points as consideration for providing the provided object. The remuneration information is information indicating the remuneration received by the broadcasting user in response to the provision of the provided object.

[0044] <Video distribution and viewing methods> Next, a method for distributing and viewing a video will be described. When the management server 11 receives a distribution request and identification information of the specified virtual space, it generates a room (distribution slot) in the virtual space specified by the user 13.

[0045] The distribution slots in which videos are currently being distributed are displayed on a list screen displayed on each user device 12. The viewing user 13B can start viewing the video by selecting the distribution slot they want to view from the list screen. The management server 11 transmits data (asset data) for displaying the virtual space to the user device 12.

[0046] Video distribution and viewing methods can be divided into client rendering, browser rendering, video distribution, and server distribution, depending on the rendering device and method. Other methods may be used as long as either the user device 12 or the management server 11 can perform image processing. In this embodiment, the client rendering method is used.

[0047] In the client rendering method, each user device 12 renders objects by executing a program. User device 12 stores 3D model data of avatar objects and objects other than avatar objects. Objects other than avatar objects are also called physical objects. Physical objects include provided objects. 3D model data of avatar objects includes data on body parts that make up the avatar object itself and on attachment parts that can be attached to the avatar object. Data on body parts includes polygon data, skeletal data (bones) for expressing the movement of the avatar object, texture data, etc. Attachment parts include texture data, etc. Broadcasting user 13A can set attachment parts for the avatar object corresponding to him / her according to his / her preferences. Model data of physical objects other than avatar objects includes polygon data, texture data, etc.

[0048] The output control unit 17 performs drawing using video data including motion data and identification information (part IDs) indicating parts of each object. The output control unit 17 also transmits video data including motion data, coordinates of the avatar object and virtual camera, etc. to the management server 11. The output control unit 17 also transmits part IDs indicating each part of the avatar object to the management server 11 at a predetermined timing (for example, at the start of distribution). The management server 11 transmits the video data to the viewing user device 12B, etc. The viewing user device 12B receives this data from the management server 11. The viewing user device 12B then uses the received data to draw images of the avatar object and physical object of the broadcasting user 13A.

[0049] The output control unit 17 also transmits audio data based on the audio collected by the microphone to the management server 11. The management server 11 transmits the audio data to the viewing user device 12B. A timestamp is attached to the motion data, audio data, etc.

[0050] The output control unit 17 of the viewing user device 12B receives video data and audio data for displaying video from the management server 11. Then, the output control unit 17 uses the video data to draw an image including an avatar object corresponding to the broadcast user 13A.

[0051] The output control units 17 of the broadcasting user device 12A and the viewing user device 12B also receive data such as messages posted by other users from the management server 11. The output control unit 17 generates output data by combining rendered images of avatar objects and other objects with posted messages, notifications, etc., and outputs the output data to the display 109. The output control unit 17 also synchronizes audio data with video data based on a timestamp and outputs the audio data from the speaker.

[0052] The broadcaster user device 12A and the viewer user device 12B may display video in different ways. For example, the broadcaster user device 12A may display video in a client-rendering manner, while the viewer user device 12B may display video in a browser-rendering manner. The user may also be able to select the video display method. For example, some of the viewer user devices 12B may display video in a client-rendering manner, while others may display video in a browser-rendering manner.

[0053] <Virtual Space> An example of virtual space 30 will be described with reference to Fig. 7. Virtual space 30 includes a plurality of objects. The objects located in virtual space 30 include an avatar object 41 and a physical object. Avatar object 41 is included in the broad category of character object 40. Fig. 7 is a diagram showing avatar object 41 and the like as viewed from above.

[0054] A world coordinate system 31 is set in the virtual space 30, indicating positions in the virtual space. The world coordinate system 31 is a coordinate system having an X-axis, a Y-axis, and a Z-axis. The world coordinate system 31 is also called a global coordinate system. The position of each object can be specified by coordinates in the world coordinate system 31. The coordinates specified in the world coordinate system 31 include coordinates (X, Y, Z) expressed in a Cartesian coordinate system. These coordinates may also include rotation coordinates (roll, pitch, yaw) relative to each coordinate axis. Each object has its own object coordinate system 32 set. The object coordinate system 32 is a coordinate system having an X-axis, a Y-axis, and a Z-axis. The orientation, movement, etc. of the avatar object 41 can be specified by coordinates in the Cartesian coordinate system or rotation coordinates.

[0055] A virtual camera 34 that changes its field of view based on input operations by the broadcasting user 13A is positioned in the virtual space 30. A camera coordinate system 33 is set for the virtual camera 34. For example, the camera coordinate system 33 includes coordinates (X, Y, Z) specified in a three-dimensional Cartesian coordinate system, as well as rotational coordinates (Xθ, Yθ, Zθ) centered on each coordinate axis.

[0056] The output control unit 17 performs rendering based on the coordinates of objects included in a predetermined field of view range Z1 (for example, a field of view cone having an approximately conical shape) centered on a virtual optical axis X1 of the virtual camera 34. Then, the output control unit 17 displays the image generated by rendering on the display 109. The rendering here refers to a drawing process that includes obtaining the position of the virtual camera 34, perspective projection, hidden surface removal based on depth information associated with each object, shading, texture mapping, etc.

[0057] At least one of the position and direction of the virtual camera 34 can be changed based on an input operation by the distribution user 13A. In this embodiment, both the position and direction of the virtual camera 34 can be changed. The virtual space 30 may include an area where the virtual camera 34 can be placed and an area where it cannot be placed.

[0058] The output control unit 17 sets the virtual camera 34 based on an input operation by the broadcast user 13A. If the avatar object 41 is included in the field of view range Z1, an image of the avatar object 41 can be displayed on the screen.

[0059] The camera direction and operations such as panning and tilting are specified by the camera coordinate system 33. Furthermore, the virtual camera 34 may be able to zoom in and out based on an input operation by the broadcast user 13A.

[0060] In this embodiment, virtual camera 34 may be associated with avatar object 41 so as to maintain a positional relationship with avatar object 41. For example, output control unit 17 may cause virtual camera 34 to follow the moving avatar object 41 during a period when no camera operation is being performed. In this case, virtual camera 34 may be set behind avatar object 41 so that an image of the back of avatar object 41 and an image of virtual space 30 in front of avatar object 41 are displayed.

[0061] In this case, the output control unit 17 acquires the movement direction indicated on the screen by the broadcasting user 13A as a first direction. The output control unit 17 converts the first direction into a second direction, which is a movement direction in the world coordinate system 31, and moves the avatar object 41 in the second direction. The output control unit 17 also sets the virtual camera 34 at a position that is a predetermined relative direction based on the avatar object 41 and a certain offset distance away. When the avatar object 41 moves, the output control unit 17 moves the virtual camera 34 in the second direction while maintaining a constant relative distance between the avatar object 41 and the virtual camera 34. This prevents the avatar object 41 from moving away from the virtual camera 34, thereby preventing the avatar object 41 from disappearing from the screen or being displayed so small that it is difficult to view. Even when the virtual camera 34 is made to follow the avatar object 41, the user 13 can change at least one of the position and direction of the virtual camera 34 by input operations.

[0062] Additionally, the virtual camera 34 may be set at a position designated by the user 13 within the allowable area, regardless of the position of the avatar object 41 . <Provided object> The provision objects 42 may be displayed in response to display requests from multiple viewing users 13B. Therefore, if a large number of provision objects 42 are continuously displayed on the screen, the provision objects 42 may occupy most of the screen. As a result, the essential avatar object 41 may be hidden by the provision objects 42. For this reason, many of the provision objects 42 are displayed dynamically. For example, the output control unit 17 may erase the provision objects 42 after displaying them for a predetermined time, or may cause the provision objects 42 to disappear from the screen after being displayed on the screen.

[0063] However, even if the provision objects 42 are dynamically displayed, some of the provision objects 42 or the avatar objects 41 may be difficult to see on the screen depending on the number and size of the provision objects 42. For this reason, the output control unit 17 displays the provision objects 42 in separate display areas according to the display attributes of the provision objects 42.

[0064] In this embodiment, the display attributes of the provided object 42 are set to be a fixed display object, an intermediate display object, an attached object, a rear display object, and a two-dimensional object. The display attributes are included in the provided object information 25. The output control unit 17 sets the display position of the provided object 42 within a display area corresponding to the display attribute of the provided object 42.

[0065] The fixed display object 43 is displayed in a predetermined fixed display area 44 within the area in which the provided object 42 can be displayed. The fixed display objects 43 may include objects that are larger in display size than other types of provided objects 42. This prevents this type of fixed display object 43 from obscuring other objects. The fixed display objects 43 may include objects that are larger in display size than the avatar object 41. An example of this type of fixed display object 43 is wallpaper.

[0066] The fixed display objects 43 may include objects that require more consideration than the other provided objects 42. Furthermore, the fixed display objects 43 may include objects that are placed in the virtual space 30 for a long time.

[0067] Examples of the fixed display object 43 include an object resembling wallpaper, and an object resembling a wall hanging (hanging scroll). The fixed display object 43 may also include objects that are intended to be placed side by side. Specifically, the fixed display object 43 may include glasses for stacking in a tower or pyramid shape, a flower stand for displaying flowers on a stand, and the like. The fixed display object 43 may display the account name of the viewing user who selected it. In the fixed display area 44, display positions 44A to 44E of the fixed display objects 43 are set together with their display order. In FIG. 7, the display position 44A at the center is first in the display order.

[0068] When the avatar object 41 is included in the field of view range Z1 of the virtual camera 34, the intermediate display object 45 is an object that is displayed between the position of the virtual camera 34 and the position of the avatar object 41 in the depth direction parallel to the optical axis X1 of the virtual camera 34. In other words, when the avatar object 41 is included in the field of view range Z1 of the virtual camera 34, the intermediate display object 45 is displayed at a position closer to the virtual camera 34 than the avatar object 41. The display position of the intermediate display object 45 is set to a position where the relative distance from the avatar object 41 is a predetermined distance L1 (offset distance). The intermediate display object 45 is a three-dimensional object. Displaying the intermediate display object 45 at the predetermined distance L1 from the avatar object 41 may mean, for example, that when a plurality of intermediate display objects 45 are displayed together as an object group, the distance to one of the intermediate display objects 45 is the predetermined distance L1. In this case, the plurality of intermediate display objects 45 may be displayed with their center points spaced apart from each other by a separation distance. Alternatively, in the case of a large intermediate display object 45, the distance from the avatar object 41 to a predetermined point (such as the center point or center of gravity) of the intermediate display object 45 may be the predetermined distance L1.

[0069] A provided object 42 that is preferably displayed together with an avatar object 41 is classified as an intermediate display object 45. The intermediate display object 45 is displayed closer to the avatar object 41 than other provided objects 42. For example, the intermediate display object 45 is an object with which the avatar object 41 can interact. The interaction refers to the avatar object 41 holding, touching, or throwing the intermediate display object 45. When expressing the interaction of the avatar objects 41, displaying the target provided object 42 near the avatar object 41 allows for a more natural movement to be expressed. As an example of this aspect, the output control unit 17 identifies a specified intermediate display object 45 based on a position specified by the broadcasting user 13A on the touch panel. At this time, the output control unit 17 may move the avatar object 41 so that its hand reaches the position of the specified intermediate display object 45. The output control unit 17 then associates the avatar object 41 with the identified intermediate display object 45. For example, the output control unit 17 makes the intermediate display object 45 a child object that has a parent-child relationship with the hand of the avatar object 41. This allows the intermediate display object 45 to follow the movement of the avatar object 41 .

[0070] 7, the intermediate display object 45 may be displayed at any position within a semicircular area 48 centered on the avatar object 41. The radius of the area 48 is a predetermined distance L1. The intermediate display object 45 may be displayed at a position on the optical axis X1 of the virtual camera 34, and away from the avatar object 41 by the predetermined distance L1.

[0071] A component that exhibits physical behavior may be set to the intermediate display object 45. As a result, after being displayed in the virtual space 30, the intermediate display object 45 falls due to the set gravity. Furthermore, if a part of the intermediate display object 45 overlaps with a part of another intermediate display object 45 or with the avatar object 41, the intermediate display object 45 collides with the other intermediate display object 45. When multiple intermediate display objects 45 are displayed simultaneously in a video, the positions at which they appear may be adjusted so that they do not overlap.

[0072] 8 , even if the avatar object 41 is not included in the field of view Z1 of the virtual camera 34, the intermediate display object 45 is displayed at a position between the position of the virtual camera 34 and the position of the avatar object 41, at a predetermined distance L1 from the avatar object 41. For example, the predetermined distance L1 between the providing object 42 and the avatar object 41 may be shorter than the distance between the providing object 42 and the virtual camera 34.

[0073] The broadcasting user 13A basically adjusts the position and orientation of the virtual camera 34 so as to display the avatar object 41. For this reason, even if the avatar object 41 is no longer included in the field of view Z1 of the virtual camera 34 as shown in Fig. 8, this state is expected to be temporary. Therefore, if the output control unit 17 displays the intermediate display object 45 at the above position, the broadcasting user 13A can recognize the intermediate display object 45 when he moves the virtual camera 34 so as to display the avatar object 41.

[0074] Although not shown, the attachment object is an object associated with a body part of the avatar object 41. For example, if the attachment object is a sword object, the attachment object is associated with the hand of the avatar object 41. As a result, the avatar object 41 is displayed as if holding the attachment object in its hand. Furthermore, when displaying the attachment object, the output control unit 17 may cause the avatar object 41 to automatically perform an action associated with the attachment object. For example, this action may be an action in which the avatar object 41 swings the attachment object, which is a sword, from top to bottom. Furthermore, the attachment object may be a clothing object. For example, the attachment object may be clothing such as a T-shirt, or accessories such as a hat. Furthermore, the attachment object may be the facial expression of the avatar object 41.

[0075] The output control unit 17 may display the attachment object as an attachment candidate rather than immediately associating it with a body part of the avatar object 41. The broadcast user selects one of one or more attachment candidates. The output control unit 17 accepts the selection operation. Then, the output control unit 17 identifies the body part, etc. of the avatar object 41 set as an attribute of the selected attachment candidate, and displays the attachment candidate in association with the identified body part, etc.

[0076] As shown in FIG. 7, the rear display object 46 is displayed at a position behind the avatar object 41 in the depth direction (Z direction) based on the virtual camera 34. The Z direction is the direction away from the virtual camera 34. In other words, the rear display object 46 is displayed at a position farther away from the position of the avatar object 41 as viewed from the virtual camera 34. The rear display object 46 may perform a preset action. For example, the action may be an action of moving diagonally downward. As a result, for example, a "shooting star" rear display object 46 is displayed as if it is flowing diagonally downward.

[0077] The two-dimensional object 49 is an object that can be represented by two-dimensional coordinates. In FIG. 7, the two-dimensional object 49 is displayed large for convenience, but is small enough not to obscure the avatar object 41. The two-dimensional object 49 is displayed within the field of view Z1 at a predetermined distance L2 (offset distance) from the virtual camera 34. The output control unit 17 always displays the two-dimensional object 49 on the screen, including it within the field of view Z1 of the virtual camera 34. Specifically, the output control unit 17 may set a hidden planar object in the virtual space 30 for displaying the two-dimensional object 49, and display the two-dimensional object 49 on the hidden planar object. Alternatively, the output control unit 17 may specify the field of view Z1 of the virtual camera 34 and display the two-dimensional object 49 within the field of view Z1.

[0078] Even if the avatar object 41 is not included in the field of view Z1 of the virtual camera 34, the output control unit 17 displays the two-dimensional object 49 as if it were displayed on the screen. The two-dimensional object 49 is, for example, a stamp or the like that indicates support for the streamer, an evaluation of the video, etc. This is because the two-dimensional object 49 is a direct message to the streamer, and therefore it is desirable for it to be displayed on the screen at all times.

[0079] The display time of the intermediate display object 45 (first object) may be longer than the display time of the two-dimensional object 49 (second object). The display time of the intermediate display object 45 can be adjusted by controlling the physical behavior. For example, the display time can be extended by slowing down the falling speed of the intermediate display object 45, making it bounce, making its movement trajectory spiral, or making its movement trajectory meander (wind).

[0080] The price of the intermediate display object 45 (first object) may be higher than the price of the two-dimensional object 49 (second object). One of the reasons for this is that it takes more time and effort for a developer to create a three-dimensional object using an information processing device than it does to create a two-dimensional object.

[0081] <Operation> The process of displaying the provided object 42 by the user device 12 will be described. For example, video distribution is performed in a distribution room created for each distribution user 13A. The distribution room is a space partitioned by walls and a floor. Alternatively, video distribution may be performed in a world such as a virtual exhibition hall or live concert venue shared by an unspecified number of users 13.

[0082] The user device 12 receives drawing data for drawing each object, such as the avatar object 41 and the provided object 42, from the management server 11 at a predetermined timing and stores the drawing data in the storage 104. The timing for receiving the drawing data is not particularly limited, and may be when logging in, when starting to watch a video, when starting to distribute a video, when receiving a display request, etc. In the following description, it is assumed that the user device 12 has received the drawing data in advance.

[0083] After the distribution of the video starts, the setting unit 15 of the broadcasting user device 12A sets the virtual camera 34 to the position and orientation specified by the broadcasting user 13A based on an input operation by the broadcasting user 13A. In addition, the communication unit 16 of the broadcasting user device 12A transmits the position and orientation of the virtual camera 34 to the management server 11. The setting unit 15 of the viewing user device 12B sets the virtual camera 34 based on the position and orientation received from the management server 11.

[0084] The viewing user device 12B displays a list of the providing objects 42 on the display 109 based on an input operation by the viewing user 13B. The viewing user device 12B accepts a selection operation by the viewing user 13B and selects a providing object 42. Based on the input operation by the viewing user 13B, the communication unit 16 transmits identification information of the providing object 42 selected by the viewing user and a display request to the management server 11. At this time, the viewing user device 12B may transmit identification information of the broadcasting user 13A, identification information assigned for each broadcast, or identification information of the room or world.

[0085] The management server 11 transmits information including the identification information of the provided object 42, the display request, and the identification information of the viewing user 13B who instructed the transmission of the display request to each user device 12. The management server 11 may also perform consideration processing and reward processing for the provided object 42. If the management server 11 determines that the provided object 42 selected by the viewing user 13B requires consideration, the management server 11 updates the user management information 24 to subtract the consideration. If the management server 11 determines that the provided object 42 is one that will grant a reward to the distribution user 13A, the management server 11 updates the user management information 24 to add the reward.

[0086] 9, the communication unit 16 of the user device 12 acquires information including a display request for the provided object 42 from the management server 11 (step S1). The acquired information includes the display request, as well as identification information of the provided object 42 and identification information of the viewing user 13B.

[0087] When receiving the display request for the provided object 42, the output control unit 17 determines the display attributes of the provided object 42 (step S2). The display attributes may be received by the communication unit 16 together with the identification information of the provided object 42. Alternatively, the output control unit 17 may determine the display attributes using information about the provided object 42 stored in the storage 104.

[0088] The output control unit 17 displays the provided object 42 at a position according to the display attributes (step S3). After identifying an area according to the display attributes, the output control unit 17 may identify one position within the area as the appearance position. Alternatively, as described above, the output control unit 17 may identify one position according to the display attributes as the appearance position. Furthermore, the output control unit 17 notifies the viewing user 13B of the provision of the provided object 42 based on the received identification information of the viewing user 13B.

[0089] <Screen display> An example of a screen displayed by the user device 12 will be described with reference to Fig. 10. Fig. 10 shows an example of a virtual space screen 60 displayed on the display 109 by the broadcast user device 12A. GUI elements such as buttons operated by the broadcast user 13A are omitted from the illustration. The virtual space screen (viewing screen) displayed by the viewing user device 12B is substantially the same as the virtual space screen 60 displayed by the broadcast user device 12A, except that its GUI elements differ from those for the broadcast user 13A.

[0090] In the screen illustrated in FIG. 10, a virtual camera 34 (omitted from FIG. 10) is set to the right of the avatar object 41. When the avatar object 41 is included in the field of view range Z1, the intermediate display object 45 is displayed in front of the avatar object 41, that is, between the virtual camera 34 and the avatar object 41. "In front" means in front of the position of the avatar object 41 as seen from the virtual camera 34, and does not necessarily mean the front where the face of the avatar object 41 is located. In other words, the relative directions of the virtual camera 34 and the avatar object 41 do not matter. The intermediate display object 45, which performs physical behavior, falls from its appearance position. The fallen intermediate display object 45 lands on the floor or the like.

[0091] The rear display object 46 is displayed behind the avatar object 41. "Rear" refers to the rear of the position of the avatar object 41 as seen from the virtual camera 34, and does not necessarily mean the rear where the back of the avatar object 41 is located.

[0092] The two-dimensional object 49 is displayed in front of the intermediate display object 45 and the rear display object 46. The two-dimensional object 49 is displayed so as to be included in the field of view Z1 of the virtual camera 34. The wearing object is displayed in association with a part of the avatar object 41, such as a T-shirt associated with the upper body shown in FIG.

[0093] The virtual space screen 60 also includes a notification 61. The notification 61 is either automatically generated by the management server 11 or posted by the viewing user 13B. The notification 61 includes the name of the viewing user 13B's account and the name of the provided object 42 provided by that viewing user 13B. This allows the distribution user 13A to express gratitude to or call out to that viewing user 13B.

[0094] The effects of the first embodiment will be described. (1-1) The user device 12 sets the display position of the provided object 42 according to the position of the virtual camera 34 and the position of the avatar object 41 corresponding to the broadcasting user 13A, based on the identification information and display request of the provided object 42 transmitted by the viewing user device 12B. Therefore, even if the position and orientation of the virtual camera 34 are changed by the broadcasting user 13A, the intermediate display object 45 and the rear display object 46 are displayed at positions based on the position of the virtual camera 34 and the position of the avatar object 41, making it easy for the broadcasting user 13A to notice the provided object 42. Therefore, the broadcasting user 13A and the viewing user 13B, who share a virtual space, can smoothly interact with each other through the display request of the provided object 42.

[0095] (1-2) The output control unit 17 sets the display position of the provided object 42 within a display area corresponding to the display attribute associated with the identification information of the provided object 42. Therefore, even if the user device 12 receives display requests for a large number of provided objects 42 within a short period of time, the provided objects 42 are displayed in different display areas according to the display attribute, thereby preventing the provided objects 42 from overlapping and becoming difficult to see.

[0096] (1-3) The output control unit 17 sets the display position of the intermediate display object 45, which is one of the providing objects 42, between the position of the virtual camera 34 and the position of the avatar object 41. This allows the intermediate display object 45 to be displayed near the avatar object 41, regardless of where the avatar object 41 is placed in the virtual space 30. Furthermore, since the virtual camera 34 is likely to be pointed at the avatar object 41, the intermediate display object 45 is likely to be displayed on the screen. This makes it easier for the distribution user 13A to notice the providing object 42.

[0097] (1-4) The output control unit 17 sets the display position of the intermediate display object 45, which is one of the provided objects 42, to a position that is a predetermined distance L1 from the avatar object 41. Therefore, by shortening the predetermined distance L1, it becomes easier for both the avatar object 41 and the provided object 42 to be included in the screen.

[0098] (1-5) The output control unit 17 sets the display position of the two-dimensional object 49, which is one of the provided objects 42, to a position within the field of view Z1 of the virtual camera 34 and at a predetermined distance L2 from the virtual camera 34. Therefore, the two-dimensional object 49 is always displayed on the virtual space screen 60.

[0099] (1-6) The output control unit 17 sets the display position of the rear display object 46, which is one of the providing objects 42, behind the avatar object 41. By dividing the display areas of the intermediate display object 45 and the rear display object 46 in this way, the providing objects 42 can be displayed without overlapping each other.

[0100] [Second embodiment] The second embodiment of the information processing system, the program, and the information processing method will be described below.

[0101] In the second embodiment, the broadcast user device 12A performs collaborative broadcasting in which multiple broadcast users 13A perform broadcasting together. Hereinafter, the same parts as in the first embodiment will be assigned the same reference numerals and detailed description thereof will be omitted.

[0102] Collaborative distribution will now be described. Distribution of a video including images of avatar objects 41 corresponding to multiple broadcast users 13A is called collaborative distribution. In this embodiment, each broadcast user device 12A can be a broadcast source device.

[0103] 11, virtual cameras 34A and 34B are set for avatar objects 41A and 41B, respectively. Each broadcast user 13A can set the position and orientation of the virtual cameras 34A and 34B. In other words, the image displayed on the display 109 of each broadcast user device 12A may differ depending on the broadcast user device 12A.

[0104] When viewing a video, the viewing user device 12B can select the distribution user 13A. The viewing user device 12B generates a virtual space image based on the position and orientation of the virtual camera 34 set by the distribution user 13A selected by the viewing user 13B.

[0105] Alternatively, all viewing user devices 12B viewing a video of a collaborative distribution may generate a virtual space image according to the position and orientation of a virtual camera 34 set by a predetermined single distribution user 13A. In this case, the distribution user 13A who sets up the virtual camera is called the host user, and the other distribution users 13A are called guest users.

[0106] (operation) The operation of the audiovisual user device 12B when providing the provision object 42 will now be described.

[0107] The communication unit 16 of the viewing user device 12B transmits information capable of identifying the broadcast user 13A to whom the provided object is to be provided, along with the identification information and display request for the provided object 42, to the management server 11. This causes the broadcast user device 12A used by that broadcast user 13A to set the display position of the provided object 42 according to the position of the virtual camera 34 and the position of the avatar object 41 corresponding to the broadcast user 13A.

[0108] This process will be described in detail. The viewing user 13B transmits identification information of the selected provision object 42 and a display request to the management server 11. At this time, the communication unit 16 transmits information capable of identifying the broadcast user 13A who is the target of the provision object 42. This information need not necessarily be a user ID as long as it can identify the broadcast user 13A. For example, this information may be information capable of identifying a broadcast room. In other words, the viewing user 13B provides the provision object 42 to the broadcast user 13A selected by the viewing user 13B.

[0109] The management server 11 transmits information including the identification information of the broadcast user 13A, the identification information of the provided object 42, and a display request to the user devices 12 that are the transmission targets. The user devices 12 that are the transmission targets may be all broadcast user devices 12A and all viewing user devices 12B. Alternatively, the user devices 12 that are the transmission targets may be only the broadcast user device 12A of the broadcast user 13A and the viewing user device 12B of the viewing user 13B that watches the video broadcast by that broadcast user 13A.

[0110] The communication unit 16 of the broadcast user device 12A acquires information including the identification information of the broadcast user 13A, the identification information of the provided object 42, the display request for the provided object 42, and the identification information of the viewing user 13B who made the display request from the management server 11 (step S1). Furthermore, upon receiving the display request for the provided object 42, the communication unit 16 identifies the display attributes of the provided object 42 (step S2).

[0111] The output control unit 17 displays the provided object 42 at a position according to the display attributes (step S3). The output control unit 17 identifies the broadcast user 13A based on the identification information of the broadcast user 13A received in step S1. The output control unit 17 then displays the intermediate display object 45 and the rear display object 46 based on the position of the avatar object 41 corresponding to the identified broadcast user 13A and the position of the virtual camera. Also, as in the first embodiment, the output control unit 17 displays the two-dimensional object 49 and the attachment object based on the avatar object 41 to be displayed.

[0112] For example, suppose that when a first broadcast user 13A and a second broadcast user 13A are broadcasting, a first viewing user 13B who is watching the video broadcast by the first broadcast user 13A makes a display request for a provided object 42. In this case, the intermediate display object 45 provided by the first viewing user 13B is displayed between the position of the virtual camera 34 set by the first broadcast user 13A and the position of the avatar object 41 corresponding to the first broadcast user 13A. Similarly, the intermediate display object 45 provided by the second viewing user 13B is displayed between the position of the virtual camera 34 set by the second broadcast user 13A and the position of the avatar object 41 corresponding to the second broadcast user 13A.

[0113] The viewing user 13B may be able to change at least one of the position and orientation of the virtual camera 34. In this case, the intermediate display object 45 provided by the first viewing user 13B is displayed between the position of the virtual camera 34 set by the first viewing user 13B and the position of the avatar object 41 corresponding to the first broadcast user 13A, which is the display target of the intermediate display object 45. Similarly, the intermediate display object 45 provided by the second viewing user 13B is displayed between the position of the virtual camera 34 set by the second viewing user 13B and the position of the avatar object 41 corresponding to the second broadcast user 13A.

[0114] As a result, the viewing user device 12B of the first viewing user 13B can display not only the provided object 42 provided to the first distribution user 13A, but also the provided object 42 provided to the second distribution user 13A if it is included in the field of view Z1 of the virtual camera 34.

[0115] According to the second embodiment, in addition to the effects (1-1) to (1-6) of the first embodiment, the following effects can be obtained. (2-1) In the second embodiment, the viewing user device 12B transmitted, together with a display request, information capable of identifying the distributor user 13A to whom the provision object 42 is to be provided to the management server 11. This allows the distributor user 13A to recognize the provision object 42 provided to them, even in collaborative distribution in which multiple avatar objects 41 are displayed on the screen.

[0116] [Third embodiment] The third embodiment of the information processing system, the program, and the information processing method will be described below.

[0117] In the third embodiment, the broadcast user device 12A broadcasts video of a game played by the broadcast user 13A. In this embodiment, the management server 11 executes the process of managing the progress of the game, but this may also be executed by a server other than the management server 11. Hereinafter, parts similar to those in the first embodiment will be assigned the same reference numerals and detailed description thereof will be omitted.

[0118] The broadcast user device 12A executes game processing based on the operation of the broadcast user 13A in a virtual space in which a plurality of objects are arranged. The type of game is not particularly limited. For example, the game may be a role-playing game, a shooting game, an action game, a racing game, a fighting game, a training simulation game, a fishing game, a love simulation game, a puzzle game, a card game, a sports game, a rhythm game, etc.

[0119] 12, the broadcast user device 12A changes at least one of the position and orientation of the virtual camera 34 based on the operation of the broadcast user 13A, as in the first embodiment. The broadcast user device 12A changes the position of a game character object 47 corresponding to the broadcast user 13A based on the operation of the broadcast user 13A. The game character object 47 is included in the broad definition of character object 40. The output control unit 17 of the broadcast user device 12A generates an image including the character object 47 and other objects.

[0120] The video distribution method can be one of the methods described above. For example, in the client rendering method, the broadcasting user device 12A transmits game progress information to the management server 11 in addition to the coordinates of each object, as in the first embodiment. The game progress information may include at least one of identification information of a reached or cleared (achieved) stage, score, parameters associated with the character object, parameters associated with the user, identification information of the opponent, action information of the character object or opponent, and win / loss. Note that the character object may reflect the movements of the broadcasting user 13A. In this case, the broadcasting user device 12A transmits motion data to the management server 11. Alternatively, the character object may not reflect the movements of the broadcasting user 13A. In this case, the output control unit 17 moves the character object based on the broadcasting user 13A's operation on the input device 110, and transmits information indicating the movement of the character object to the management server 11. The management server 11 transmits information or motion data indicating the movement of the character object to the viewing user device 12B. Alternatively, the broadcaster user device 12A may transmit the generated video to the viewer user device 12B via the management server 11.

[0121] (operation) The operation of the viewing user device 12B when providing a providing object 42 will be described. An example of the providing object 42 in this embodiment is an object that can affect the broadcast user 13A's play of the game. For example, the providing object 42 is an item used in the game. The effect of the providing object 42 may be advantageous or disadvantageous to the broadcast user 13A when playing the game. Alternatively, another example of the providing object 42 is an object that does not directly affect the broadcast user 13A's play of the game, such as a message to the broadcast user 13A.

[0122] The communication unit 16 of the viewing user device 12B transmits identification information of the provision object 42 selected by the viewing user 13B and a display request to the management server 11. At this time, the viewing user device 12B may transmit identification information of the viewing user 13B to the management server 11.

[0123] The management server 11 transmits the received identification information of the broadcast user 13A, the identification information of the provided object 42, and the display request to each user device 12. As shown in FIG. 9, the communication unit 16 of the user device 12 receives a display request for the provided object 42 from the management server 11 (step S1).

[0124] When the communication unit 16 receives the display request for the providing object 42, it identifies the display attribute of the providing object 42 (step S2). The providing object 42 in the game may be any one of an intermediate display object 45, an attachment object, a rear display object 46, and a two-dimensional object 49.

[0125] The output control unit 17 displays the provided object 42 at a position according to the display attribute (step S3). For example, the output control unit 17 displays the intermediate display object 45 based on the position of the character object 40 and the position of the virtual camera 34.

[0126] The intermediate display object 45 may be attached to the character object 40 based on an operation by the broadcast user 13A. In other words, the intermediate display object 45 may also have the function of an attached object.

[0127] 12 , for example, if the intermediate display object 45 is a weapon item such as a sword, the output control unit 17 displays the intermediate display object 45, which is the weapon item, between the character object 40 and the virtual camera 34. When the distribution user 13A selects the intermediate display object 45, the output control unit 17 displays the weapon item in association with the hand part of the character object 40.

[0128] Furthermore, the output control unit 17 may display a providing object 42 that changes a parameter associated with the character object 40 or the user 13. In the example shown in Fig. 12, a rear display object 46 that increases the vitality value of the character object 40 is displayed. Each time this rear display object 46 is displayed, the vitality value of the character object 40 increases.

[0129] In a game in which broadcast users 13A compete against each other or against an NPC (Non-Player Character), the greater the number of display requests, the higher the probability of winning. Note that an NPC is a character that is not controlled by a player. This provides a motivation for broadcast users 13A to increase the number of viewing users.

[0130] Furthermore, when a game is played in which broadcast users 13A compete against each other, the management server 11 may pit broadcast users 13A who are expected to have a similar number of viewing users against each other. The viewing user 13B can register the broadcast user 13A as a favorite user 13A in the management server 11. The management server 11 can obtain the number of registrations of the broadcast user 13A as a favorite user 13A. The management server 11 may select as opponents broadcast users 13A whose difference in the number of registrations is within a predetermined range.

[0131] Furthermore, in a game, if the provision objects 42 are provided without any restrictions, the provision objects 42 may not be used or the game balance may become unbalanced. For this reason, as described above, a price may be set for the provision of the provision objects 42. For example, a provision object 42 that is highly effective in contributing to victory may be set with a higher price, such as a higher number of coins or points to be consumed. Alternatively, the management server 11 may set the number of times each provision object 42 can be provided. For example, the number of times a "sword" weapon item can be provided per battle may be set to "1 time," or the like. Furthermore, the number of times a rear display object 46 that increases the stamina value can be provided per viewing user 13B per battle may be set to "3 times," or the like.

[0132] According to the third embodiment, in addition to the effects (1-1) to (1-6) of the first embodiment, the following effects can be obtained. (3-1) In the third embodiment, the broadcasting user device 12A broadcasts a video of the broadcasting user 13A playing a game. The viewing user device 12B also transmits a display request for a providing object 42 that affects the game. This allows the viewing user 13B to participate in the game by cooperating (or interfering) with the broadcasting user 13A, even if the viewing user 13B cannot display his or her own character object 40 in the video. This allows the broadcasting user 13A and the viewing user 13B to smoothly interact with each other through the display request for the providing object 42 in the game.

[0133] The above-described embodiments can be modified as follows: The above-described embodiments and the following modifications can be combined with each other within the scope of technical compatibility. In the second embodiment, the viewing user 13B provides a provision object 42 to the broadcaster user 13A who is broadcasting the video being viewed. Alternatively or additionally, the viewing user 13B may specify the broadcaster user 13A to whom the provision object 42 is to be provided. The viewing user device 12B displays a screen including avatar objects 41 corresponding to multiple broadcast users 13A. In other words, the field of view of the virtual camera 34 used to display this screen may include multiple avatar objects 41. The screen includes GUI elements for selecting each broadcast user 13A. The GUI elements are buttons or menu displays corresponding to each avatar object 41, etc. The viewing user device 12B accepts an operation of the GUI elements by the viewing user 13B to select an avatar object 41. The viewing user device 12B also accepts an operation to request the display of the provision object 42. If the providing object 42 is an intermediate display object 45, the broadcasting user device 12A and the viewing user device 12B display the intermediate display object 45 between the position of the avatar object 41 specified by the viewing user 13B who provides the intermediate display object 45 and the position of the virtual camera 34. Also, if the providing object 42 is a rear display object 46, the broadcasting user device 12A and the viewing user device 12B display the rear display object 46 behind the specified avatar object 41.

[0134] Alternatively, the provided object 42 requested for display by the viewing user 13B may be assigned to the host user. In this case, the provided object 42 may be displayed based on the virtual camera 34 and avatar object 41 of the broadcast user 13A who is the host user, as in the above embodiments. Alternatively, the host user may assign the provided object 42 to his / her own avatar object 41 or to the avatar object 41 of the guest user. In this case, the viewing user device 12B transmits information including the identification information of the provided object 42 and the display request to the broadcast user device 12A of the host user via the management server 11. The broadcast user device 12A of the host user displays a GUI element (or a screen) for selecting a user to which the provided object 42 is to be assigned based on the received information. The host user selects a user 13 to which the provided object 42 is to be assigned based on the GUI element. If the host user selects himself / herself, the broadcast user device 12A of the host user transmits the identification information of the host user and the identification information of the provided object 42 to the other user device 13 via the management server 11. Each user device 13 displays the providing object 42 based on the position of the host user's avatar object 41 as needed based on the display attributes. When the host user selects a guest user other than the host user, the broadcasting user device 12A transmits the identification information of the selected user and the identification information of the providing object 42 to the other user devices 13 via the management server 11. Each user device 13 displays the providing object 42 based on the position of the guest user's avatar object 41 as needed based on the display attributes. The host user may select one or more guest users.

[0135] In the third embodiment, the broadcasting user 13A broadcasts a game video. Alternatively or additionally, the display processing of the provided object 42 may be applied to a game in which multiple users 13 participate. In this embodiment, there is no broadcast of the video, and there may not be a viewing user 13B who watches the video. The game may be a multiplayer online (MO) game. The users 13 may form a group (also called a guild or party) to progress through the game and cooperate with each other to progress through the game. Alternatively, the users 13 may be opponents fighting each other. In this embodiment, each user 13 can change the position and orientation of the virtual camera 34 on the user device 12 they use. Also in this embodiment, one user 13 provides a provided object 42 to the other user 13. The provided object 42 may be an object that supports the gameplay of the other user 13 or an object that supports the game, depending on the content of the game. These objects may be items with a specific function in the game, objects that increase or restore parameters associated with the user 13, etc. Alternatively, the provided object 42 may interfere with the game play of the other user 13 .

[0136] In the above embodiments, the user device 12 sets the display position of the provided object 42 according to the display attribute, but the management server 11 may set the display position. In this case, when the management server 11 receives a display request for the provided object 42, the management server 11 may identify the display attribute of the provided object 42 and transmit the display request for the provided object 42, the display position thereof, and identification information of the viewing user 13B to each user device 12.

[0137] The viewing user 13B may be able to select the display position of the provided object 42. As illustrated in FIG. 13, multiple candidate areas 50 may be set as areas in which the provided object 42 can be displayed. In FIG. 13, three candidate areas 50A to 50C are set. The candidate areas 50A to 50C may be the same as those in the first embodiment. That is, they may be areas between the avatar object 41 and the virtual camera 34, behind the avatar object 41, or fixed display areas that are not affected by the position of the avatar object 41. The viewing user device 12B may display the candidate areas 50A to 50C on the display 109. The viewing user device 12B accepts the selection of the provided object 42. Before or after the selection of the provided object 42, the viewing user device 12B accepts a selection operation of one of the multiple candidate areas 50A to 50C. The viewing user device 12B transmits the selected provided object 42 and the selected candidate areas 50A to 50C to the management server 11 together with a display request. When the management server 11 receives this display request, it may transmit the display request for the provided object 42, its display position, and identification information of the viewing user 13B to each user device 12. In addition, the price for the provided object 42 may be determined according to the candidate areas 50A to 50C. As an example, the price for the candidate area 50A between the avatar object 41 and the virtual camera 34 may be higher than the price for the candidate area 50B behind the avatar object 41.

[0138] <Provided object> In the above embodiment, the provided objects 42 are the fixed display object 43, the intermediate display object 45, the rear display object 46, the two-dimensional object 49, and the attached object. Alternatively, the provided objects 42 may include one of these objects. Alternatively, the provided objects 42 may include at least one of these objects and another object.

[0139] <Output control section> In each of the above embodiments, the output control unit 17 of the user device 12 distributes and views videos using the client rendering method. Alternatively, distribution and viewing may be performed using any of the above methods other than the client rendering method. The browser rendering method is a method in which the distributor user device 12A and the viewer user device 12B display videos using independent web browsers stored in the storage 104. In this case, the distributor user device 12A transmits video data and the like to the management server 11. The user device 12 displaying the videos receives web data from the management server 11, which is written in a markup language such as HTML (Hyper Text Markup Language) and uses CSS (Cascading Style Sheets), JavaScript (registered trademark), and the like. The web browser program launched on the user device 12 renders avatar objects and other objects using JavaScript running on the browser. In this method, model data of avatar objects and the like is not stored in the user device 12. The browser rendering method does not require an application to be installed on the user device 12. Alternatively, the user device 12 may have an application including an in-app browser installed.

[0140] In the browser rendering method, at least one of well-known methods such as static rendering (Static Server-Side Rendering), server-side rendering (Server-Side Rendering), client-side rendering (Client-Side Rendering), pre-rendering (CSR with Prerendering), and hydration rendering (SSR with (Re)Hydration) can be used as a method for generating a file that displays the virtual space screen 60 by rendering. Rendering here refers to interpreting data written in a markup language or the like and calculating the layout of image elements and characters to be displayed on the virtual space screen 60.

[0141] In static rendering, a web server stores static HTML files in advance. The web server may be included in the management server 11. The user device 12 downloads the static HTML files from the web server. The user device 12 displays the rendered HTML files on the display 109 to display the virtual space screen 60. In server-side rendering (SSR), the web server performs rendering and other operations in response to a request from the user device 12 and transmits the rendered HTML file to the user device 12. The user device 12 displays the downloaded HTML file on the display 109 to display the virtual space screen 60. The web server also includes an application server. In client-side rendering (CSR), the user device 12 performs rendering by executing JavaScript (registered trademark). At this time, the user device 12 downloads a simple HTML file from the web server, as well as necessary JavaScript (registered trademark), CSS files, etc. In pre-rendering, which is a type of CSR, the user device 12 generates and stores some HTML files in advance and downloads and renders the necessary content from the web server upon request. In hydration rendering, the web server generates and stores HTML files that have not yet been rendered. The user device 12 downloads the HTML file from the web server and downloads the necessary data based on the HTML file for rendering. Note that in each of the above methods, the user device 12 may obtain the necessary data from the management server 11 instead of the web server.

[0142] The video distribution method is a method in which the broadcaster user device 12A generates video data. Specifically, the broadcaster user device 12A draws objects using motion data and the like, almost similar to the image processing in the client rendering method, and generates video data. The broadcaster user device 12A also transmits the encoded video data and audio data to the management server 11. The management server 11 transmits the video data and audio data to the viewing user device 12B. The viewing user device 12B displays video on the display 109 based on the video data received from the broadcaster user device 12A.

[0143] The server distribution method is a method in which the management server 11 generates video data. The management server 11 receives motion data and audio data from the broadcasting user device 12A. The management server 11 then generates output data for outputting video based on this data. The management server 11 transmits the generated output data to the broadcasting user device 12A and the viewing user device 12B. The broadcasting user device 12A and the viewing user device 12B output video to the display 109 based on the received output data.

[0144] In this way, either the user device 12 or the management server 11 is responsible for generating data for outputting moving images. The data to be generated differs depending on the distribution method. The device responsible for generating the data can be changed depending on the distribution method.

[0145] The virtual space 30 may be a space where events such as live performances or matches are held, as long as the virtual space screen is displayed. Alternatively, the virtual space 30 may be a virtual store or an entertainment facility such as an amusement park. Alternatively, the virtual space 30 may be a space where communication between users 13 is possible.

[0146] <Administration Server> The management server 11 may be a single server device equipped with multiple virtual servers, or may be a combination of multiple physical servers. For example, the management server 11 may be a combination of a server having a distribution management unit 20, a server having a user information management unit 21, a server having a user management information storage unit 22, and a server having a provided object information storage unit 23.

[0147] <Avatar Object> In the above embodiments, the user device 12 renders the avatar object 41 corresponding to the broadcast user 13A using tracking data or emote data that detects the broadcast user 13A's movements. This allows the movements of the broadcast user 13A or the movements selected by the broadcast user 13A to be reflected in the avatar object 41. Alternatively or additionally, the user device 12 may temporarily or continuously render the avatar object 41 without using tracking data or emote data. The output control unit 17 controls the avatar object 41 based on input operations by the user 13. The avatar object 41 performs actions in accordance with instructions input by the user 13. This aspect reduces the rendering processing load on the user device 12 when rendering a large number of avatar objects 41 in the virtual space 30.

[0148] <user> In the above embodiments, the broadcasting user 13A refers to a user who transmits at least one of information related to images (video) and information related to audio. For example, the broadcasting user 13A is a user who broadcasts videos independently (first embodiment) or a user who performs collaborative broadcasting in which multiple people can participate (second embodiment). Alternatively, the broadcasting user 13A may be a user who hosts or organizes a video chat or voice chat in which multiple people can participate and / or watch, or an event (party, etc.) in a virtual space in which multiple people can participate and / or watch. In other words, the broadcasting user 13A can also be referred to as a host user, organizer user, or organizer user.

[0149] On the other hand, the viewing user 13B refers to a user who receives at least one of information related to video and information related to audio. However, the viewing user 13B not only receives the information but also reacts to the information. For example, the viewing user 13B is a user who watches a video broadcast or a collaboration broadcast. Alternatively, the viewing user 13B may be a user who participates in and / or watches a video chat, a voice chat, or an event. Therefore, the viewing user 13B can also be referred to as a guest user, a participating user, a listener, a viewing user, a supporting user, etc.

[0150] <Information Processing System> In the above embodiments, the user device 12 is an information processing device such as a smartphone, a mobile phone, a tablet terminal, a personal computer, a console game machine, or a wearable computer such as a head-mounted display. Alternatively, the information processing system may be a system installed in a video distribution studio. The information processing system includes an information processing device, a server, a sensor unit worn on the body of the distribution user 13A, a tracking system that detects the position of the sensor unit, an operation unit, a speaker, a display, etc. The tracking system may include a multi-axis laser emitter that emits pulsed laser light for synchronization. The sensor unit includes a sensor that detects laser light and detects its own position and orientation while synchronizing with the synchronization pulse.

[0151] Next, the technical ideas that can be understood from the above-described embodiment and other examples will be described below. [1] An information processing system for displaying an image of a virtual space visually recognized by a distribution user who distributes a video and a viewing user who watches the video, a broadcast user device used by the broadcast user, a setting unit that changes at least one of the position and the direction of the virtual camera in the three-dimensional virtual space; a communication unit that acquires information transmitted from a user device used by the viewing user, the information including identification information for identifying an object selected by the viewing user and a display request for the object; an output control unit that sets a display position of the object according to the position of the virtual camera and the position of a character object corresponding to the broadcasting user based on the identification information of the object and the display request, and displays the virtual space image in which the object is placed at the display position on a display device.

[0152] [2] An information processing system for displaying an image of a virtual space of a game that is viewed by multiple users, a setting unit that changes at least one of a position and a direction of a virtual camera in a three-dimensional virtual space based on an operation by the user; a display request sending unit that sends, based on an operation by one of the users, identification information for identifying an object selected by the one of the users and a display request for the object; a display request receiving unit that receives identification information for identifying the object and a display request for the object; an output control unit that sets a display position of the object according to the position of the virtual camera and the position of a character object corresponding to the first user based on the identification information of the object and the display request, and displays the virtual space image in which the object is placed at the display position on a display device.

[0153] [3] An information processing system for displaying an image of a virtual space visually recognized by a first user and a second user, a setting unit that changes at least one of the position and the direction of the virtual camera in the three-dimensional virtual space; a communication unit that acquires information transmitted from a user device used by the second user, the information including identification information for identifying an object selected by the second user, designation information for designating a position or area in which the object is to be displayed, and a display request for the object; an output control unit that sets a display position of the object according to the specified position based on the object's identification information, the specification information, and the display request, and displays a virtual space image in which the object is placed at the display position on a display device. [Explanation of symbols]

[0154] 10...Information Processing Systems 11...Administration server 12...User device

Claims

1. An information processing system for displaying an image of a virtual space visually recognized by a first user and a second user, a setting unit that changes at least one of the position and the direction of a virtual camera in the three-dimensional virtual space; a communication unit that acquires information transmitted from a user device used by the second user, the information including identification information for identifying an object selected by the second user and a display request for the object; an output control unit that sets a display position of the object according to the position of the virtual camera and the position of a character object corresponding to the first user based on the object's identification information and the display request, and displays a virtual space image in which the object is placed at the display position on a display device.

2. one or more computers, a setting unit that changes at least one of the position and the direction of the virtual camera in the three-dimensional virtual space; an output control unit that outputs a virtual space image, which is an image of the virtual space seen from the virtual camera, to a display device that is visually recognized by a user; a communication unit that acquires information transmitted from another device used by another user, the information including identification information for identifying an object selected by the other user and a display request for the object; The output control unit sets a display position of the object according to the position of the virtual camera and the position of a character object corresponding to the user based on the identification information of the object and the display request, and outputs the object to the display position.

3. a display attribute is associated with the object, and a plurality of the display attributes are associated with respective display areas; The program according to claim 2 , wherein the output control unit sets the display position of the object within the display area corresponding to the display attribute associated with the identification information of the object.

4. The program according to claim 2 , wherein the output control unit sets the display position of the object between the position of the virtual camera and the position of the character object.

5. The program according to claim 4 , wherein the output control unit sets the display position of the object to a position that is a predetermined distance relative to the character object.

6. The program according to claim 4 , wherein the output control unit sets the display position of the object to a position within a field of view of the virtual camera and at a predetermined relative distance from the virtual camera.

7. the output control unit displays a first object that is the object displayed between the position of the virtual camera and the position of the character object, and a second object that is at a predetermined relative distance from the virtual camera; 3. The computer-readable medium according to claim 2, wherein the first object is a three-dimensional object and the second object is a two-dimensional object.

8. the output control unit displays a first object that is the object displayed between the position of the virtual camera and the position of the character object, and a second object that is at a predetermined relative distance from the virtual camera; The program according to claim 2 , wherein the display time of the first object is set to be longer than the display time of the second object.

9. the output control unit displays a first object that is the object displayed between the position of the virtual camera and the position of the character object, and a second object that is at a predetermined relative distance from the virtual camera; The program according to claim 2 , wherein a price for making the display request for the first object is higher than a price for making the display request for the second object.

10. The program according to claim 2 , wherein the output control unit sets the display position of the object to a position behind the character object in a depth direction relative to the virtual camera.

11. The user is a viewing user who views a video including an image of the character object, The communication unit receiving, from a server, data for displaying the video transmitted by another device used by a distributor user who distributes the video; transmitting the identification information for identifying the object selected by the viewing user and the display request for the object to a server; The program according to claim 2 , wherein the output control unit outputs the video based on data for displaying the video.

12. the video includes images of the character objects corresponding to the plurality of broadcast users, the virtual camera is set for each of the character objects, The program of claim 11, wherein the communication unit causes the other device used by the broadcast user to set the display position of the object according to the position of the virtual camera corresponding to the broadcast user and the position of the character object by transmitting to the server, along with identification information of the object and the display request, information capable of identifying the broadcast user to whom the object is to be provided.

13. One or more computers changing at least one of the position and the direction of the virtual camera in the three-dimensional virtual space; outputting a virtual space image, which is an image of the virtual space seen from the virtual camera, to a display device that is visually recognized by a user; acquiring information transmitted from another device used by another user, the information including identification information for identifying an object selected by the other user and a display request for the object; An information processing method that sets a display position of the object according to the position of the virtual camera and the position of a character object corresponding to the user based on the identification information of the object and the display request, and outputs the object at the display position.

14. An information processing method using one or more computers for displaying an image of a virtual space visually recognized by a first user and a second user, The computer changing at least one of the position and the direction of a virtual camera in the three-dimensional virtual space; acquiring information transmitted from a user device used by the second user, the information including identification information for identifying an object selected by the second user and a display request for the object; An information processing method that sets a display position of the object according to the position of the virtual camera and the position of a character object corresponding to the first user based on the object's identification information and the display request, and displays a virtual space image in which the object is placed at the display position on a display device.

Images