Information processing system, information processing method, and information processing program

Abstract

To provide content distribution technology which is suitable for virtual spaces.SOLUTION: Disclosed is an information processing system 1 comprising: a first detection unit 211A that detects the state of a first user who distributes a content that includes the image of a first virtual space H20; a first drawing processing unit 200A that draws the image of the first virtual space which is visible to the first user via a first terminal device 20A pertaining to the first user; a first storage unit 240A that stores a first display medium that is associated with the first user. The image of the first virtual space has a plurality of layers. The first drawing processing unit includes a first drawing unit 201A that draws a first image region of a layer pertaining to the first display medium among the plurality of layers, and a second drawing unit 202A that draws a second image region of a layer pertaining to a first user interface among the plurality of layers, and the first drawing unit changes the state of the first display medium in the first image region on the basis of the state of the first user detected by the first detection unit.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present disclosure relates to an information processing system, an information processing method, and an information processing program. [Background technology]

[0002] BACKGROUND ART "Mirrativ" has been known as a service for distributing music and videos to multiple users (see, for example, Non-Patent Document 1). [Prior art documents] [Non-patent literature]

[0003] [Non-Patent Document 1] “Mirrativ”, [online], Mirrativ Inc., [Retrieved December 18, 2019], Internet (URL: https: / / www.mirrativ.com / ) Summary of the Invention [Problem to be solved by the invention]

[0004] The above-mentioned conventional technologies are services that can be viewed in real space, and it is difficult to expand them to virtual space.

[0005] Therefore, an object of the present disclosure is to provide a content distribution technology suitable for virtual spaces. [Means for solving the problem]

[0006] According to one aspect of the present disclosure, an information processing system is disclosed that includes a drawing processing unit that draws an image of a virtual space and a user interface unit that executes processing in response to various inputs in the virtual space, wherein the drawing processing unit updates the image of the virtual space having multiple layers in response to the progress of virtual reality in the information processing system and user operations, and the multiple layers include any two or more layers from a first display medium display layer on which a first display medium is displayed, a second display medium display layer on which a second display medium is displayed, an interface display layer on which a user interface is displayed, a background layer, and an auxiliary information display layer on which various auxiliary information is displayed, and wherein, based on the viewpoint of a user viewing the image of the virtual space, if the background layer is included, the background layer is set to be the backmost layer, and the anteroposterior relationship of the other layers is set according to the situation. [Effects of the Invention]

[0007] According to the present disclosure, a content distribution technique suitable for virtual space is provided. [Brief explanation of the drawings]

[0008] [Figure 1] 1 is a block diagram of a virtual reality generation system according to an embodiment of the present invention. [Figure 2] FIG. 2 is an explanatory diagram of an image viewed by each of the left and right eyes. [Figure 3] FIG. 2 is an explanatory diagram of the structure of an image in a virtual space according to the present embodiment. [Figure 4] FIG. 1 is an explanatory diagram of a virtual space corresponding to a home screen (lobby screen). [Figure 5] FIG. 2 is an explanatory diagram of a hierarchical structure of a home image. [Figure 5A] FIG. 6 is an enlarged view of the field of view from the user's viewpoint in FIG. 5 (when looking straight ahead). [Figure 6] FIG. 10 is a diagram showing an example of a home image viewed from the wearable device. [Figure 7A] FIG. 10 is an explanatory diagram of a manner in which the front and back of the operation areas are swapped. [Figure 7B]FIG. 10 is an explanatory diagram of a manner in which the front and back of the operation areas are swapped. [Figure 8A] FIG. 2 is a schematic side view of a virtual space corresponding to a distributed image. [Figure 8B] FIG. 2 is a schematic plan view of a virtual space corresponding to a delivery image as seen from above. [Figure 9] FIG. 2 is an explanatory diagram of a hierarchical structure of a delivery image. [Figure 10] FIG. 10 is an explanatory diagram showing the relationship between the mirror and each layer of the delivered image from a top view. [Figure 11] FIG. 10 is an explanatory diagram of an interface display layer related to a delivery image. [Figure 12] FIG. 10 is a diagram showing an example of a broadcast image viewed by a broadcast user via a wearable device. [Figure 13] FIG. 10 is a diagram showing an example of a distributed image viewed by a viewing user via a wearable device. [Figure 14] FIG. 10 is a diagram illustrating an example of a distribution image for a smartphone. [Figure 15] FIG. 2 is a schematic block diagram showing functions of a terminal device on the content distribution side. [Figure 16] FIG. 10 is an explanatory diagram of an example of drawing information of an avatar. [Figure 17] FIG. 2 is a schematic block diagram showing functions of a terminal device on the content viewing side. [Figure 18] FIG. 2 is a schematic block diagram showing the functions of a server device. [Figure 19] FIG. 10 is a flow diagram showing an example of operations leading up to the start of distribution of specific content. [Figure 20] FIG. 10 is a flow chart showing an example of operations leading up to the start of viewing of specific content. [Figure 21] FIG. 10 is a flow diagram showing an example of an operation while a distribution user is distributing a specific content (that is, while a viewing user is viewing the specific content). DETAILED DESCRIPTION OF THE INVENTION

[0009] Hereinafter, embodiments will be described in detail with reference to the accompanying drawings.

[0010] An overview of a virtual reality generation system 1 according to one embodiment of the present invention will be described with reference to Fig. 1. Fig. 1 is a block diagram of the virtual reality generation system 1 according to this embodiment.

[0011] The virtual reality generation system 1 includes a server device 10 (an example of an external processing device) and one or more terminal devices 20. For simplicity, three terminal devices 20 are shown in Fig. 1, but the number of terminal devices 20 may be two or more.

[0012] The server device 10 is an information processing system such as a server managed by an operator that provides one or more virtual realities. The terminal devices 20 are devices used by users, such as mobile phones, smartphones, tablet terminals, personal computers (PCs), wearable devices (head-mounted displays, eyeglass-type devices, etc.), or game devices. A plurality of terminal devices 20 can be connected to the server device 10 via the network 3, typically in different modes for each user.

[0013] The terminal device 20 is capable of executing a virtual reality application according to this embodiment. The virtual reality application may be received by the terminal device 20 from the server device 10 or a predetermined application distribution server via the network 3, or may be stored in advance in a storage device provided in the terminal device 20 or in a storage medium such as a memory card readable by the terminal device 20. The server device 10 and the terminal device 20 are communicatively connected via the network 3. For example, the server device 10 and the terminal device 20 cooperate to execute various processes related to virtual reality.

[0014] The terminal devices 20 include a terminal device 20A on the content distribution side (an example of a first terminal device) and a terminal device 20B on the content viewing side (an example of a second terminal device). In the following, for the sake of explanation, the terminal device 20A on the content distribution side and the terminal device 20B on the content viewing side are described as separate terminal devices, but the terminal device 20A on the content distribution side may also be the terminal device 20B on the content viewing side, or vice versa. In the following, when there is no particular need to distinguish between the terminal device 20A and the terminal device 20B, they may simply be referred to as "terminal device 20."

[0015] The terminal devices 20 are connected to each other so as to be able to communicate with each other via the server device 10. In the following, "one terminal device 20 transmits information to another terminal device 20" means "one terminal device 20 transmits information to another terminal device 20 via the server device 10". Similarly, "one terminal device 20 receives information from another terminal device 20" means "one terminal device 20 receives information from another terminal device 20 via the server device 10". However, in a modified example, the terminal devices 20 may be connected to each other so as to be able to communicate with each other without going through the server device 10.

[0016] The network 3 may include a wireless communication network, the Internet, a Virtual Private Network (VPN), a Wide Area Network (WAN), a wired network, or any combination of these.

[0017] 1, the virtual reality generation system 1 includes studio units 30A and 30B. The studio units 30A and 30B are content distribution side devices, similar to the content distribution side terminal device 20A. The studio units 30A and 30B may be located in a content production studio, room, hall, or the like.

[0018] Each studio unit 30 may have the same functions as the terminal device 20A on the content distribution side and / or the server device 10. For simplicity of explanation, the following mainly describes a case where the terminal device 20A on the content distribution side distributes various contents to the terminal devices 20B on each content viewing side via the server device 10. However, instead of or in addition to this, the studio units 30A and 30B facing the distribution users may have the same functions as the terminal device 20A on the content distribution side, thereby distributing various contents to the terminal devices 20B on each content viewing side via the server device 10. In a modified example, the virtual reality generation system 1 may not include the studio units 30A and 30B.

[0019] In the following, the virtual reality generation system 1 realizes an example of an information processing system, but each element of a specific terminal device 20 (see the terminal communication unit 21 to the terminal control unit 25 in FIG. 1) may realize an example of an information processing system, or multiple terminal devices 20 may cooperate to realize an example of an information processing system. Also, the server device 10 may independently realize an example of an information processing system, or the server device 10 and one or more terminal devices 20 may cooperate to realize an example of an information processing system.

[0020] Here, an overview of virtual reality according to this embodiment will be described. The virtual reality according to this embodiment is a virtual reality for any reality, such as education, travel, role-playing, simulation, and entertainment such as games and concerts, and a virtual reality medium such as an avatar is used in conjunction with the execution of the virtual reality. For example, the virtual reality according to this embodiment is realized by a three-dimensional virtual space, various virtual reality media appearing in the virtual space, and various contents provided in the virtual space.

[0021] Virtual reality media are electronic data used in virtual reality, and include any media, such as cards, items, points, in-service currency (or virtual reality currency), tickets, characters, avatars, parameters, etc. Virtual reality media may also be virtual reality-related information such as level information, status information, parameter information (such as stamina and attack power), or ability information (such as skills, abilities, spells, and jobs). Virtual reality media are electronic data that can be acquired, owned, used, managed, exchanged, synthesized, enhanced, sold, discarded, or donated by users within virtual reality, but the manner of use of virtual reality media is not limited to those explicitly described in this specification.

[0022] In this embodiment, the users include viewing users (an example of a second user) who view various contents in the virtual space, and broadcasting users (an example of a first user) who broadcast specific contents described later in the virtual space via a broadcaster avatar M2 (an example of a first display medium) described later.

[0023] Note that a broadcasting user may, as a viewing user, view specific content from other broadcasting users, and conversely, a viewing user may, as a broadcasting user, broadcast specific content. However, in the following, for the sake of simplicity, a viewing user will be referred to as the viewing user at the time, and a broadcasting user will be referred to as the broadcasting user at the time. Note that in the following, when there is no particular distinction between a broadcasting user and a viewing user, they may simply be referred to as "users." Furthermore, when there is no particular distinction between a viewer avatar M1 (an example of a second display medium) and a broadcaster avatar M2 related to a viewing user, they may simply be referred to as "avatars."

[0024] An avatar typically has the form of a character facing forward, and may have the form of a human, an animal, or the like. Avatars can have a variety of appearances (appearances when drawn) by being associated with various avatar items.

[0025] Basically, the viewing user and the distribution user wear a wearable device on, for example, their head or part of their face, and view the virtual space through the wearable device. The wearable device may be a head-mounted display or a glasses-type device. The glasses-type device may be so-called AR (Augmented Reality) glasses or MR (Mixed Reality) glasses. In either case, the wearable device may be separate from the terminal device 20, or may realize some or all of the functions of the terminal device 20. In the present embodiment, as an example, the terminal device 20 is realized by a head-mounted display.

[0026] The following description will mainly focus on specific content by a distribution user among various types of content distributed by the server device 10. The following description will also focus on content that is suitable for viewing via a head-mounted display.

[0027] Specific content by the broadcasting user refers to content in which a broadcaster avatar M2 related to the broadcasting user appears in a virtual space, and the broadcaster avatar M2 changes its orientation, position, movement, etc. depending on the orientation, position, movement, etc. of the broadcasting user. Note that the orientation, position, and movement of the broadcasting user is a concept that includes not only the orientation, position, and movement of the broadcasting user's face, hands, or any part or all of their body, but also the orientation, position, movement, or the like of the broadcasting user's gaze. Specific content by the broadcasting user is typically video content.

[0028] The specific content provided by the broadcast user typically provides entertainment in any manner via the broadcaster avatar M2. For example, the specific content provided by the broadcast user may be related to various performances such as dance, music, magic, etc., chats, meetings, gatherings, conferences, or the like.

[0029] Furthermore, the specific content provided by the broadcasting user may be educational. For example, the specific content provided by the broadcasting user may include instruction or advice from the broadcasting user via the broadcaster avatar M2. For example, content provided in virtual reality related to dance lessons may include instruction or advice from a dance teacher. In this case, the dance teacher becomes the broadcasting user, and the students become viewing users, allowing the students to receive individual instruction from the teacher in virtual reality.

[0030] In addition, specific content by a distribution user may include a collaboration between two or more distribution users (hereinafter abbreviated as "collaboration"). This enables distribution in a variety of ways and promotes interaction between distribution users.

[0031] The server device 10 can also distribute content other than the specific content provided by the distribution user. The types and number of content (content provided in virtual reality) provided by the server device 10 are arbitrary. In the present embodiment, as an example, the content provided by the server device 10 may include digital content such as various types of video. The video may be real-time video or non-real-time video. The video may also be video based on real images or video based on CG (Computer Graphics). The video may be video for providing information. In this case, the video may be related to an information providing service in a specific genre (a service providing information on travel, housing, food, fashion, health, beauty, etc.), a broadcasting service by a specific user (for example, YouTube (registered trademark)), etc.

[0032] The manner in which content is provided in virtual reality is diverse and may be other than by using the display function of a head-mounted display. For example, if the content is a video, the content may be provided by rendering the video on a display device (virtual reality medium) in the virtual space. The display device in the virtual space may take any form, such as a screen installed in the virtual space, a large-screen display installed in the virtual space, or a display on a mobile device in the virtual space.

[0033] Additionally, virtual reality content may be viewable by methods other than a head-mounted display. For example, virtual reality content may be viewed directly (without a head-mounted display) via a smartphone, tablet, or the like.

[0034] (Server device configuration) The configuration of the server device 10 will be specifically described. The server device 10 is configured by a server computer. The server device 10 may be realized by a plurality of server computers working together. For example, the server device 10 may be realized by a server computer that provides various types of content, a server computer that functions as various authentication servers, and the like working together. The server device 10 may also include a web server. In this case, some of the functions of the terminal device 20, which will be described later, may be realized by a browser processing HTML documents received from the web server and various programs (Javascript) associated with the HTML documents.

[0035] As shown in FIG. 1, the server device 10 includes a server communication unit 11, a server storage unit 12, and a server control unit 13.

[0036] The server communication unit 11 includes an interface for communicating with an external device wirelessly or via a wired connection to send and receive information. The server communication unit 11 may include, for example, a wireless LAN (Local Area Network) communication module or a wired LAN communication module. The server communication unit 11 is capable of sending and receiving information to and from the terminal device 20 via the network 3.

[0037] The server storage unit 12 is, for example, a storage device, and stores various information and programs required for various processes related to virtual reality.

[0038] The server control unit 13 may include a dedicated microprocessor or a CPU (Central Processing Unit) that implements a specific function by loading a specific program, a GPU (Graphics Processing Unit), etc. For example, the server control unit 13 cooperates with the terminal device 20 to execute a virtual reality application in response to a user operation on the display unit 23 of the terminal device 20.

[0039] (Terminal Device Configuration) The following describes the configuration of the terminal device 20. As shown in Fig. 1, the terminal device 20 includes a terminal communication unit 21, a terminal storage unit 22, a display unit 23, an input unit 24, and a terminal control unit 25.

[0040] The terminal communication unit 21 includes an interface for communicating with an external device wirelessly or via a wired connection to transmit and receive information. The terminal communication unit 21 may include a wireless communication module, a wireless LAN communication module, or a wired LAN communication module that supports mobile communication standards such as LTE (Long Term Evolution) (registered trademark), LTE-A (LTE-Advanced), a fifth-generation mobile communication system, or UMB (Ultra Mobile Broadband). The terminal communication unit 21 can transmit and receive information to and from the server device 10 via the network 3.

[0041] The terminal storage unit 22 includes, for example, a primary storage unit and a secondary storage unit. For example, the terminal storage unit 22 may include a semiconductor memory, a magnetic memory, an optical memory, or the like. The terminal storage unit 22 stores various information and programs used in virtual reality processing received from the server device 10. The information and programs used in virtual reality processing may be acquired from an external device via the terminal communication unit 21. For example, a virtual reality application program may be acquired from a predetermined application distribution server. Hereinafter, the application program may also be simply referred to as an application.

[0042] The terminal storage unit 22 also stores data for rendering virtual spaces, such as images of indoor spaces such as buildings, outdoor spaces, etc. Note that multiple types of data for rendering virtual spaces may be prepared for each virtual space and used separately.

[0043] The terminal storage unit 22 also stores various images (texture images) to be projected (texture mapping) onto various objects arranged in the three-dimensional virtual space.

[0044] For example, the device storage unit 22 stores drawing information of a viewer avatar M1 as a virtual reality medium associated with each user. The viewer avatar M1 is drawn in the virtual space based on the drawing information of the viewer avatar M1.

[0045] The device storage unit 22 also stores drawing information of a broadcaster avatar M2 as a virtual reality medium associated with each broadcast user. The broadcaster avatar M2 is drawn in the virtual space based on the drawing information of the broadcaster avatar M2.

[0046] The device storage unit 22 also stores drawing information related to various objects different from the viewer avatar M1 and the broadcaster avatar M2, such as various gift objects, buildings, walls, or NPCs (Non Player Characters). Various objects in the virtual space are drawn based on such drawing information. A gift object is an object corresponding to a gift from one user to another user, and is a part of an item. The gift object may be something worn by the avatar (clothes or accessories), something to decorate the broadcast image (fireworks, flowers, etc.), a background (wallpaper), or the like, or a ticket or the like that can be used to spin a gacha (lottery).

[0047] The display unit 23 includes a display device such as a liquid crystal display or an organic EL (Electro-Luminescence) display. The display unit 23 is capable of displaying a variety of images. The display unit 23 is configured, for example, with a touch panel, and functions as an interface that detects a variety of user operations. As described above, the display unit 23 may be built into a head-mounted display.

[0048] The input unit 24 may include physical keys or any other input interface, such as a pointing device like a mouse. The input unit 24 may also be capable of accepting non-contact user input, such as voice input, gesture input, or gaze input. Gesture input may utilize sensors (such as image sensors, acceleration sensors, and distance sensors) for detecting various user states, dedicated motion capture integrating sensor technology and cameras, or controllers like joypads. The gaze detection camera may also be located within a head-mounted display. As described above, various user states may include, for example, the user's orientation, position, movement, or the like. In this case, the user's orientation, position, and movement are concepts that include not only the orientation, position, and movement of the user's hands and body, but also the gaze direction, position, movement, or the like of the broadcasting user.

[0049] The terminal control unit 25 includes one or more processors and controls the overall operation of the terminal device 20.

[0050] The terminal control unit 25 transmits and receives information via the terminal communication unit 21. For example, the terminal control unit 25 receives various information and programs used for various processes related to virtual reality from at least one of the server device 10 and another external server. The terminal control unit 25 stores the received information and programs in the terminal storage unit 22. For example, the terminal storage unit 22 may store a browser (Internet browser) for connecting to a web server.

[0051] The terminal control unit 25 launches a virtual reality application in response to a user operation. The terminal control unit 25 executes various processes related to virtual reality in cooperation with the server device 10. For example, the terminal control unit 25 causes the display unit 23 to display an image of a virtual space. For example, a GUI (Graphical User Interface) that detects a user operation may be displayed on the screen. The terminal control unit 25 can detect a user operation via the input unit 24. For example, the terminal control unit 25 can detect various operations performed by a user's gestures (operations corresponding to a tap operation, a long tap operation, a flick operation, a swipe operation, etc.). The terminal control unit 25 transmits operation information to the server device 10.

[0052] The terminal control unit 25 draws the broadcaster avatar M2 and the like together with the virtual space (image) and displays them on the display unit 23. In this case, for example, as shown in FIG. 2, a stereoscopic image may be generated by generating images G200 and G201 that are viewed by the left and right eyes, respectively. FIG. 2 schematically illustrates images G200 and G201 that are viewed by the left and right eyes, respectively. Note that, hereinafter, unless otherwise specified, the image of the virtual space refers to the entire image represented by images G200 and G201. Furthermore, the terminal control unit 25 realizes various movements of the broadcaster avatar M2 in the virtual space in response to various operations by the broadcasting user, for example. Specific drawing processing by the terminal control unit 25 will be described later.

[0053] FIG. 3 is an explanatory diagram of the structure of an image in a virtual space according to this embodiment.

[0054] In this embodiment, the image of the virtual space has multiple layers 300, as conceptually shown in Fig. 3. In this case, the terminal control unit 25 generates the image of the virtual space by drawing the image area of ​​each layer and overlaying the image areas of each layer. Note that in the example shown in Fig. 3, the layer 300 consists of four layers 301 to 304, but the number of layers is arbitrary.

[0055] The multiple layers may include a broadcasting avatar display layer J0 on which the broadcaster avatar M2 is displayed, a user avatar display layer J1 on which the viewer avatar M1 is displayed, an interface display layer J2 on which the user interface is displayed, a background layer J3, an auxiliary information display layer J5 on which various auxiliary information is displayed, or a combination of any two or more of such layers.

[0056] In the following, the front-to-back (or behind) relationship of multiple layers is based on the viewpoint of the user viewing the image in the virtual space, and the distance relationship between each layer is based on the distance along the line of sight of the user viewing the image in the virtual space.

[0057] If the multiple layers include a background layer J3, the background layer J3 will be the backmost layer, but the chronological order of the other layers may be set as appropriate. For example, if the multiple layers relating to an image of a virtual space include a distribution avatar display layer J0, an interface display layer J2, a background layer J3, and an auxiliary information display layer J5, the layers may include, from front to back, the distribution avatar display layer J0, the interface display layer J2, the auxiliary information display layer J5, and the background layer J3.

[0058] The interface display layer J2 is a layer related to the above-mentioned input unit 24 and is capable of receiving contactless user input. The interface display layer J2 is preferably placed within reach of the user in the virtual space (i.e., within operable distance). This allows the user to perform various inputs without moving around in the virtual space while wearing the head-mounted display, and allows the user to perform various inputs via the interface display layer J2 in a manner that is less likely to cause motion sickness.

[0059] The auxiliary information display layer J5 is preferably disposed behind the interface display layer J2. In this case, the auxiliary information display layer J5 is disposed out of reach of the user in the virtual space (unreachable without moving). This improves the visibility of the interface display layer J2 compared to when the auxiliary information display layer J5 is disposed in front of the interface display layer J2.

[0060] The auxiliary information displayed on the auxiliary information display layer J5 is optional, but may include, for example, at least one of input information from a viewing user of a specific content (i.e., a viewing user who is viewing the specific content), guidance / notification information for the distribution user and / or other users, and item information including a gift object to be given to the distribution user.

[0061] The auxiliary information display layer J5 may be composed of multiple layers, with different layers being formed for each attribute of auxiliary information. For example, the auxiliary information display layer J5 on which the gift object is drawn may be set separately from the other auxiliary information display layers J5. In this case, the auxiliary information display layer J5 on which the gift object is drawn may be positioned behind the distribution avatar display layer J0 and closer to it than the other auxiliary information display layers J5. Alternatively, the auxiliary information display layer J5 on which the gift object is drawn may be positioned in front of the distribution avatar display layer J0, different from the other auxiliary information display layers J5.

[0062] The background layer J3 may have the function of defining the outer boundary (shell) of the virtual space. The virtual space may be an infinite space that does not substantially have a background layer J3, or may be a cylindrical or spherical space.

[0063] (Example of virtual reality) The server control unit 13 cooperates with the terminal device 20 to display an image of the virtual space on the display unit 23, and updates the image of the virtual space in response to the progress of the virtual reality and the user's operations.

[0064] The drawing process described below is realized by the terminal device 20, but in other embodiments, part or all of the drawing process described below may be realized by the server control unit 13. For example, in the following description, at least part of the image of the virtual space displayed on the terminal device 20 may be a web display that is displayed on the terminal device 20 based on data generated by the server device 10, and at least part of the screen may be a native display that is displayed by a native application installed on the terminal device 20.

[0065] Here, some examples of images of the virtual space generated by the virtual reality generation system 1 will be described with reference to FIGS.

[0066] FIG. 4 is an explanatory diagram of a virtual space corresponding to a home screen (lobby screen). FIG. 5 is an explanatory diagram of the hierarchical structure of an image of the home screen (hereinafter referred to as "home image H1"). FIG. 5A is an enlarged view of the field of view R500 from the user's viewpoint in FIG. 4 (when looking straight ahead). FIG. 6 is a diagram showing an example of the home image H1 viewed from a head-mounted display. FIGS. 7A and 7B are explanatory diagrams of how the front and back of the operation area are swapped.

[0067] The home image H1 represents a virtual space (hereinafter referred to as the "home space") that serves as an entrance for moving to various virtual spaces. FIG. 4 shows a user located within the home space. As will be described later, the home space has an operation area G300 for selecting various content. From the home space, users can perform various activities, such as viewing specific content from a desired broadcasting user, broadcasting specific content as a broadcasting user, or broadcasting specific content in collaboration with other users (hereinafter also referred to as "collaborative broadcasting").

[0068] In the present embodiment, as an example, the home image H1 includes, from the front, an interface display layer J2 (an example of a layer related to the second user interface), an auxiliary information display layer J5, and a background layer J3. Note that in modified examples, other layers such as those described above may be included, or some layers (for example, the auxiliary information display layer J5 or the background layer J3) may be omitted.

[0069] 5 shows an example of the arrangement of the interface display layer J2, auxiliary information display layer J5, and background layer J3 in a top view along a predetermined reference axis I passing through the user's position (the position of the viewer avatar M1 or the broadcaster avatar M2). In this specification, the top view refers to a top view based on the up-down direction of the virtual space, unless otherwise specified.

[0070] In this embodiment, as shown in FIG. 5 , the interface display layer J2, the auxiliary information display layer J5, and the background layer J3 are arranged in a top view around a predetermined reference axis I to form cylindrical surfaces with different radii r1, r2, and r3, where each of the radii r1, r2, and r3 corresponds to the distance between the corresponding layer and the user. Note that the radius r3 of the background layer J3 may be infinite (infinitely far away). Furthermore, as described above, the radius r1 of the interface display layer J2 may be set within the reach of the user's hand. Furthermore, the radius r1 may be adjustable for each user. This allows the interface display layer J2 to be formed at a distance that suits the length of the user's hand and their preferences.

[0071] The interface display layer J2 is preferably positioned based on the user's position within the home space. In other words, when the user enters the home space, the user is positioned in a predetermined positional relationship with respect to the interface display layer J2. In this embodiment, the interface display layer J2 is positioned within reach of the user based on the user's position within the home space. This allows the user to perform various inputs while wearing the head-mounted display without moving within the virtual space, and allows the user to perform various inputs via the interface display layer J2 in a manner that is less likely to cause motion sickness.

[0072] The interface display layer J2 preferably includes a plurality of planar operation areas G300 that function as the input unit 24. In this case, the plurality of planar operation areas G300 can function as selection buttons that can be selected by the user. The size and shape of the planar operation areas G300 are arbitrary and may vary depending on the attributes of the operation areas G300. This makes it easier for the user to distinguish the operation areas G300 for each attribute.

[0073] In this case, the various processes (examples of various second processes) realized by operating the operation area G300 are arbitrary, but may include, for example, a process for changing the layout of the user interface (e.g., multiple planar operation areas G300), a process for moving the user to any location or a specific location in the virtual space (e.g., a process for moving the broadcast user to a broadcast location), a process for starting the distribution of various content, a process for starting viewing of various content, a process for ending viewing of various content, a voice input process, a process for giving a gift to the broadcast user, a lottery process for avatar items, a process for selecting / exchanging avatar items, a character input process, and a process for transitioning to a state in which any of these processes can be executed, or any combination thereof. This allows the user to realize a variety of operations via the interface display layer J2.

[0074] The multiple planar operation areas G300 associated with the content viewing start process may be drawn in a manner that associates the contents of the selection candidate content, etc., with each other. For example, each operation area G300 may be drawn with a thumbnail image or real-time video of the selection candidate content (e.g., specific content by a broadcasting user) (see FIG. 4). Furthermore, the thumbnail of a collaborative broadcast may be drawn with broadcaster avatars M2 of the multiple broadcasting users who are performing the collaborative broadcast. This allows the user to easily understand what content is available for viewing and easily select a desired operation area G300 (e.g., an operation area G300 related to a desired specific content).

[0075] Furthermore, the plurality of planar operation areas G300 are preferably arranged in a form of a plurality of rows, as shown in Fig. 4. This allows for efficient arrangement of many operation areas within the user's viewable range, even if the number of planar operation areas G300 increases due to, for example, an increase in the number of distribution users (and a corresponding increase in the number of specific contents being distributed).

[0076] In this case, at least a portion of the multiple planar operation areas G300 are arranged in multiple rows along a first curved surface 501 (see FIGS. 4 and 5A) around a predetermined reference axis I. The predetermined reference axis I is an axis that passes through the user's position and extends in the vertical direction of the home space. In this case, in a top view of the home space, the first curved surface 501 may have a circular shape that defines the interface display layer J2, and the center of curvature may be located on the predetermined reference axis I. However, in a modified example, the first curved surface 501 may have an elliptical shape or a similar shape in a top view of the home space. This allows the user to view the multiple planar operation areas G300 at equal distances from the user, thereby improving operability. Note that, "arranging the planar operation areas G300 along the first curved surface 501" may mean that, in a top view of the home space, the in-plane direction of the planar shape of the operation area G300 is parallel to the tangent direction of the first curved surface 501. Alternatively, if the radius of curvature of first curved surface 501 is sufficiently small, operation area G300 may be projected onto first curved surface 501.

[0077] Furthermore, the multiple planar operation areas G300 may be preferably arranged in a manner grouped by category. For example, an operation area G300 related to the user's "recommended" category may be arranged in an area along the first curved surface 501 that is in front of the user's viewpoint, and different categories (e.g., "Collaboration Town," "Game," "Beginner," "Follow," etc.) may be arranged on the left and right sides of the front area. The arrangement of each operation area G300 may be customized for each user in a manner similar to the arrangement of icons for various apps on a smartphone screen. In FIG. 4, the area in front of the user's viewpoint is schematically shown as area R500 (see FIG. 5). The home image H1 is generated based on the area in front of the user's viewpoint. Note that the position of area R500 in front of the user's viewpoint may change as the user's line of sight or facial orientation changes. This improves consistency between changes in the field of view in real space and changes in the field of view in virtual space.

[0078] Furthermore, the multiple planar operation areas G300 are preferably arranged in a front-to-back layered configuration. For example, the multiple planar operation areas G300 may include a first group arranged in multiple rows along a first curved surface 501 about a predetermined reference axis I, and a second group arranged in multiple rows along a second curved surface 502 about the predetermined reference axis I. In this case, the second curved surface 502 may be offset behind the first curved surface 501, as shown in FIG. 4. In this case, when viewed from the user's viewpoint, the multiple operation areas G300 of the second group arranged along the second curved surface 502 (hereinafter also referred to as "operation area G300-2" to distinguish from the first group) overlap behind the multiple operation areas G300 of the first group arranged along the first curved surface 501 (hereinafter also referred to as "operation area G300-1" to distinguish from the second group). In this case, when viewed from the user's viewpoint, the second group of operation areas G300-2 may be partially visible from behind the first group of operation areas G300-1. In this case, it is possible to let the user know of the existence of the operation areas behind and to efficiently increase the number of operation areas that can catch the user's eye. Note that a third curved surface offset further behind the second curved surface may be set, and an additional planar operation area G300 may be arranged. In this way, any number of two or more planar operation areas G300 may be arranged in an overlapping manner when viewed from the user's viewpoint.

[0079] Furthermore, in the case of multiple planar operation areas G300 that overlap one another, it is possible to efficiently reduce the processing load related to drawing while arranging the multiple operation areas G300 so that they can be operated by the user. For example, it is possible to reduce the processing load related to drawing as a whole by performing complete drawing of thumbnail images and real-time video only in operation area G300-1 and performing incomplete drawing (e.g., processing such as changing the texture) in operation area G300-2. From a similar perspective, it is also possible to reduce the processing load related to drawing as a whole by performing complete drawing of thumbnail images and real-time video only in operation area G300-1 within area R500 in front of the user's viewpoint, while performing incomplete drawing (e.g., processing such as changing the texture) in operation area G300-1 outside the front area R500. For example, by pre-caching or pre-loading data that is likely to become operation area G300-1 in the front area R500 in the user's terminal device 20, it is possible to reduce latency while efficiently reducing the number of requests submitted via the network 3, the volume of requests imposed on the network 3 accordingly, and the computational resources used to respond to the requests. In this case, the data that is likely to become operation area G300-1 in the front area R500 may be predicted based on the tendency of each user, or may be determined by machine learning based on artificial intelligence, etc.

[0080] 5A shows a schematic top view of the relationship between the first curved surface 501 and the second curved surface 502. The offset distance between the first curved surface 501 and the second curved surface 502 may be significantly smaller than the distance between layers with different attributes (for example, the distance between the interface display layer J2 and the auxiliary information display layer J5). This makes it easier for the user to understand that they are the same interface display layer J2, and also makes it easier for the user to intuitively understand that multiple planar operation areas G300 exist on the back side.

[0081] In the case of the interface display layer J2 in which such a plurality of planar operation areas G300 are arranged, the user may be able to change the arrangement of the user interface (e.g., the plurality of planar operation areas G300) by a specific operation, which allows the arrangement of the user interface (e.g., the plurality of planar operation areas G300) according to the user's preferences and tastes.

[0082] For example, some of the multiple planar operation areas G300 may function as buttons for changing the arrangement of the multiple planar operation areas G300. Alternatively, the user may move the multiple planar operation areas G300 left and right as viewed from the user's viewpoint by a gesture input of moving the hand left and right. FIG. 6 shows a state in which the left operation area G300 (denoted as "G300L" in FIG. 6) is in the process of moving to the front as viewed from the user's viewpoint due to a user operation. In this way, the multiple planar operation areas G300 may be moved left and right along the first curved surface 501. This makes it possible to easily change the operation area G300 located in the front as viewed from the user's viewpoint among the multiple planar operation areas G300. In this case, the multiple planar operation areas G300 may be moved in such a way that the planar operation area G300 located in the front area changes for each category. This allows the grouping of each category to be maintained, thereby achieving both ease of viewing and operability.

[0083] Alternatively, some or all of the multiple planar operation areas G300 may be continuously, periodically, or irregularly moved left and right along the first curved surface 501 (and / or the second curved surface 502), or may be moved in a circular manner. Such a movement may be appropriately changeable based on a setting by the user, or may be realized based on an event in which a predetermined movement condition is met.

[0084] Furthermore, in the case of multiple planar operation areas G300 overlapping one another as described above, the user may be able to swap the order of the operation areas by performing a predetermined input. For example, as described above, when viewed from the user's viewpoint, if a first group of multiple operation areas G300-1 arranged along the first curved surface 501 is overlapped behind a second group of multiple operation areas G300-2 arranged along the second curved surface 502, the user may be able to swap the first and second groups by performing a predetermined input by moving their hand in a predetermined manner, as schematically shown in FIG. 7A. This enables intuitive operation and improves operability. In this case, the second group of multiple operation areas G300-2 that were arranged along the second curved surface 502 are now arranged along the first curved surface 501 as the first group, and the first group of multiple operation areas G300-1 that were arranged along the first curved surface 501 are now arranged along the second curved surface 502 as the second group. Note that such a front-back swapping operation may be applied only to the operation area G300 located in front of the user among the plurality of planar operation areas G300. This reduces the processing load compared to swapping all of the plurality of planar operation areas G300.

[0085] The front-rear swapping may be performed in such a manner that the entire first group of multiple operation areas G300-1 arranged along the first curved surface 501 rotates backward as shown by the arrows in Fig. 7A, or in such a manner that the upper portions of the first group of multiple operation areas G300-1 arranged along the first curved surface 501 rotate backward from the top and the lower portions rotate backward from the bottom, as shown by the arrows in Fig. 7B. Conversely, the first and second groups may be switched in such a manner that the upper portions of the second group of multiple operation areas G300-2 arranged along the second curved surface 502 rotate forward from the top and the lower portions rotate forward from the bottom, as shown by the arrows in Fig. 7B. Note that although two groups have been described here, the same applies to the case of three or more groups (i.e., multiple planar operation areas G300 overlapping in three or more layers).

[0086] In this embodiment, the home image H1 does not include the user avatar display layer J1, but this is not limited to this. For example, the home image H1 may include the user avatar display layer J1 on the closer side (closer to the user's viewpoint) than the interface display layer J2. In this case, the user avatar display layer J1 of the home image H1 may be drawn with hands, for example, when the user extends their hands forward. This allows the user to perform operations on the multiple planar operation areas G300 while watching the hand movements, thereby improving operability.

[0087] 8A and 8B are explanatory diagrams of a virtual space corresponding to a distributed image, where FIG. 8A is a schematic side view and FIG. 8B is a schematic plan view from above. In FIGS. 8A and 8B, the distribution user is indicated by the symbol 808, and the position of the corresponding distributor avatar M2 is indicated by a dotted-line box. FIG. 9 is an explanatory diagram of the hierarchical structure of an image of specific content by the distribution user (hereinafter referred to as the "distributed image H2"). FIG. 10 is an explanatory diagram showing the relationship between the mirror and each layer of the distributed image H2 from a top view. FIG. 11 is an explanatory diagram of an interface display layer J2 related to the distributed image H20 for the distribution user. FIG. 12 is a diagram showing an example of the distributed image H2 viewed by the distribution user through a head-mounted display (an example of a first wearable device). FIG. 13 is a diagram showing an example of the distributed image H2 viewed by a viewing user through a head-mounted display (an example of a second wearable device). FIG. 14 is a diagram showing an example of the distributed image H2 for a smartphone. The delivered image H2 shown in FIG. 14 is an image that can be viewed without a head-mounted display, and can also be used for similar devices other than smartphones.

[0088] The distribution image H2 corresponds to an image of a virtual space where the distributor avatar M2 is placed, such as a virtual space for the distributor user to create specific content (hereinafter referred to as a "content production space") or a similar virtual space. The content production space may correspond to a space such as a studio in the real world, for example. The distributor user can carry out production activities of specific content in the content production space.

[0089] In this embodiment, the delivered image H2 includes a delivered image H20 for the delivery user (see FIG. 12), a delivered image H21 for the viewing user that is viewable via a head-mounted display (see FIG. 13), and a delivered image H22 for the viewing user that is viewable without a head-mounted display (hereinafter also referred to as a "smartphone delivered image H22") (see FIG. 14). Hereinafter, unless otherwise specified, the delivered image H2 corresponds to any of these delivered images.

[0090] In the present embodiment, as an example, the delivery image H2 includes, from the front, a delivery avatar display layer J0, an interface display layer J2 (an example of a layer related to the first user interface), an auxiliary information display layer J5, a user avatar display layer J1, and a background layer J3. Note that in modified examples, other layers such as those described above may be included, or some layers (e.g., the auxiliary information display layer J5 or the background layer J3) may be omitted. Furthermore, the interface display layer J2 and the auxiliary information display layer J5 may be integrated as appropriate.

[0091] Figure 9 shows an example arrangement of the broadcasting avatar display layer J0, interface display layer J2, auxiliary information display layer J5, user avatar display layer J1, and background layer J3 in a top view, along with a predetermined reference axis I passing through the position of the broadcasting user (the position of the broadcaster avatar M2).

[0092] In this embodiment, as shown in FIG. 9 , the distribution avatar display layer J0, interface display layer J2, auxiliary information display layer J5, user avatar display layer J1, and background layer J3 are arranged in a top view to form vertical planes at different distances d1 to d5 along the user's line of sight passing through a predetermined reference axis I. In this case, the distances d1 to d5 correspond to the distance between the respective layers and the user. Note that the distance d5 related to the background layer J3 may be infinity or a distance corresponding to the plane farthest from the virtual camera's viewpoint (far clip plane). In this case, the interface display layer J2 may be set to the plane closest to the virtual camera's far clip plane (near clip plane). Note that in this case, the distribution user's viewpoint and the virtual camera may be set at the same position as the face or head object of the broadcaster avatar M2, and the optical axis of the virtual camera may be set in a direction facing forward from the viewpoint of the broadcaster avatar M2. In this case, the face and head of the broadcaster avatar M2 are not drawn from the broadcasting user's viewpoint, but the arms and hands of the broadcaster avatar M2 (the movement of the hands and arms when extending one's hands out in front of the broadcaster avatar M2 to operate the broadcaster avatar M2) may be drawn. Even in this case, the broadcaster avatar M2 may be drawn as the broadcast avatar display layer J0 by other users (e.g., viewing users). Furthermore, the distance d2 of the interface display layer J2 may be set within the reach of the user's hand, as described above. Furthermore, the distance d2 may be adjustable for each user. Furthermore, the distance d2 may be the same as the radius r1 (see FIG. 5). This allows the interface display layer J2 to be formed with a sense of distance that suits the length and preferences of the user's hand. Furthermore, when the distance d2 is the same as the radius r1 (see FIG. 5), similar (common) operability can be achieved between the broadcasting user's broadcast image H20 and the home image H1, thereby improving convenience.

[0093] The content production space, which is the virtual space in which the broadcaster avatar M2 is active, is arbitrary, and for example, in the example shown in FIGS. 8A and 8B , it may include a broadcasting room 800, which may include a closet 801 for the broadcaster avatar M2 to change clothes and a table 802. The broadcasting user can select a desired avatar item from the avatar items in the closet 801 to prepare (standby) the broadcaster avatar M2. Note that an avatar item is an item drawn in association with an avatar such as the broadcaster avatar M2, and may include, for example, hairstyle, clothing, accessories, skin color, etc.

[0094] In this embodiment, as a preferred example of the content production space, a mirror 805 is placed in the broadcasting room 800. The mirror 805 has the property of reflecting light (visible light), similar to a mirror in the real world. The image reflected in the mirror 805 corresponds to the broadcasting image H20. Therefore, by positioning the broadcaster avatar M2 corresponding to the broadcasting user in front of the mirror 805, the broadcasting user can check the various states of the broadcaster avatar M2 (and the corresponding states of the broadcasting image H20) corresponding to the broadcasting user's various states (orientation, position, and movement) while looking straight ahead (at the mirror 805). This allows the broadcasting user to check the various states of the broadcaster avatar M2 (and the corresponding states of the broadcasting image H20) corresponding to the broadcasting user's various states (orientation, position, and movement) and the corresponding states of the broadcaster avatar M2 (and the corresponding states of the broadcasting image H20) while facing the mirror 805 (facing forward) and keeping their gaze directed toward the mirror 805. In this way, the distribution user can check their own movements while facing the mirror 805 and correct their own movements in real time as needed, so that they can easily generate the desired distribution image H20 that matches their own image.

[0095] 10 shows a top view of the positional relationship between mirror 805 in broadcasting room 800, the broadcasting user, and each layer of broadcast image H2. In FIG. 10, area R502 corresponds to area R501 shown in FIG. 9, and area R502, which corresponds to the field of view through mirror 805 when viewed from the front from the broadcasting user's viewpoint, corresponds to the field of view of a virtual camera, assuming that a virtual camera is placed in mirror 805. In this case, the virtual camera in mirror 805 can fulfill the same function as mirror 805 by displaying a captured image on mirror 805.

[0096] A broadcaster avatar is drawn on the broadcast avatar display layer J0. In this case, a broadcaster avatar M2 is drawn on the broadcast avatar display layer J0 in a state corresponding to various states (orientation, position, and movement) of the broadcaster user standing in front of the mirror 805. Note that the various states (orientation, position, and movement) of the broadcaster user can be acquired via the input unit 24, as described above.

[0097] The interface display layer J2 is preferably positioned based on the position of the broadcasting user standing in front of the mirror 805. In other words, when the broadcasting user stands in front of the mirror 805, the broadcasting user is positioned in a predetermined positional relationship with respect to the interface display layer J2. In this embodiment, the interface display layer J2 is positioned within reach of the broadcasting user when standing in front of the mirror 805, based on the position of the broadcasting user. Note that the predetermined positional relationship may be changeable by the user as appropriate. This makes it possible to realize an interface display layer J2 that suits the user's preferences and physique.

[0098] The interface display layer J2 preferably includes one or more operation areas that function as input units 24. Various operation areas can function as selection buttons that can be selected by the broadcast user. For example, the broadcast user may be able to select (operate) a desired selection button via contactless user input.

[0099] In this case, the various processes (examples of various first processes) realized by operating the operation area are arbitrary, but may include at least one of, for example, a process for starting distribution of specific content, a process for ending distribution of specific content, a voice input process, a process for the distribution user to receive a gift, a lottery process for items (e.g., items related to avatars), a process for selecting / exchanging various items, a character input process, an approval process for other users, a process for changing the values ​​of parameters of the distribution content (e.g., various parameters of the distribution image H2 described below), and a process for transitioning to a state in which these processes can be executed.

[0100] 11 shows an example of an operation area G100 in the form of various icons in a distribution image H20 for a distributor user. In the example shown in FIG. 11, the various operation areas G100 include an operation area G101 for chatting (comments), an operation area G102 for taking screenshots, an operation area G103 for playing games, an operation area G104 for sharing other video content (including sharing the same video between a viewing user and a distributor user), an operation area G105 for spinning a gacha (lottery), etc. Note that the item to be drawn by the gacha is arbitrary, and may be, for example, an avatar item.

[0101] The operation area in the interface display layer J2 may be displayed all the time or only in specific cases, thereby enabling operations via the operation area in the interface display layer J2 while expanding the display area of ​​the broadcaster avatar M2 to display the entire broadcaster avatar M2.

[0102] Furthermore, a specific operation area on the interface display layer J2 may be displayed in a manner similar to the operation area G300 related to the home image H1 as described above. In this case, the various operation areas on the interface display layer J2 may be rearranged in the manner described above with reference to Figures 7A and 7B.

[0103] In a preferred embodiment, the operation area in the interface display layer J2 includes an operation area G120 (hereinafter also referred to as the "smartphone small window area G120") for adjusting various parameters of the delivery image H2. The smartphone small window area G120 may display part or all of the delivery image H21 for the viewing user and / or part or all of the delivery image H22 for the smartphone.

[0104] The various parameters of the delivery image H2 that can be adjusted via the smartphone small window area G120 are, for example, parameters related to the virtual camera, and specifically may include brightness, distance from the virtual camera (i.e., the distance between the mirror 805 and the delivery user), optical axis direction of the virtual camera, horizontal position of the virtual camera (position relative to the delivery user in the horizontal direction intersecting the optical axis direction), zoom (magnification), etc. In this case, for example, the delivery user may be able to change the zoom of the image in the smartphone small window area G120 by, for example, pinching out or pinching in, while operating the smartphone small window area G120 as if it were a smartphone, or may be able to translate the image in the smartphone small window area G120 by swiping. Alternatively, the delivery user may be able to change the position of the virtual camera (its position relative to themselves) by performing an operation to move the position of the smartphone small window area G120. Furthermore, the broadcasting user may be able to switch between a tracking mode, in which the position of the virtual camera automatically follows the broadcasting user's movement, and a fixed mode, in which the position of the virtual camera is fixed, by controlling the state of the virtual camera via the smartphone small window area G120. In this case, the tracking mode enables mobile broadcasting. Furthermore, the broadcasting user may be able to create a state that the broadcasting user does not want to be shown to the viewing user, such as an Away From Keyboard (AFK) state or a mute state, by controlling the state of the virtual camera via the smartphone small window area G120. This prevents an increase in processing load due to the drawing of unnecessary objects, while improving convenience for the broadcasting user.

[0105] Incidentally, when a user is wearing a head-mounted display or the like, the operability when adjusting various parameters of the delivered image H2 tends to be poor, unlike operations in real space. However, by providing a small smartphone window area G120, the operability is improved.

[0106] Furthermore, when a smartphone-specific distribution image H22 is displayed in the smartphone window area G120, the distribution user can create specific content while checking the distribution image H22 as viewed on a smartphone, along with the distribution image H20 that substantially matches the distribution image H21 as viewed via a head-mounted display. Note that a small window for another viewing device, such as a tablet, may be set instead of or in addition to the smartphone window area G120. The distribution user may be able to select a desired small window to display in the distribution image H20.

[0107] In the example shown in Figure 12, a small smartphone window area G120 is displayed adjacent to a mirror 805 that reflects broadcaster avatar M2. The small smartphone window area G120 may display a broadcast image H22 (see Figure 14) as it would appear on a smartphone. In this case, the broadcasting user can broadcast while checking in real time the status of the broadcast image H22 on the terminal device 20 of the viewing user who is viewing on the smartphone.

[0108] In a modified example, the distribution image H20 may include two concepts: an image during distribution and a preparatory image before distribution. In the case of the preparatory image before distribution, an operation area G106 (see FIG. 11) for opening the closet 801 (see FIG. 8A) and changing clothes, an operation area G107 (see FIG. 11) for starting distribution, etc. may be drawn in the interface display layer J2.

[0109] The auxiliary information display layer J5 may display text information, such as gift items and support messages from viewers of the specific content related to the distribution image H2, in real time. This allows the distribution user to distribute the specific content while enjoying the reactions of viewers and interacting with them. The auxiliary information display layer J5 may also display the name or title of the distribution of the specific content. For example, in the example shown in FIG. 12, the name of the distribution G11, "Relaxed Chat Distribution," a heart-shaped gift object G12, and various comments G13, such as "This is my first time seeing this..." are displayed.

[0110] A viewer avatar M1 of a viewing user of specific content related to the distribution image H2 may be drawn on the user avatar display layer J1. This allows the distributor user to distribute while understanding the number and status of viewing users. Note that the viewer avatar M1 drawn on the user avatar display layer J1 may be drawn in a state corresponding to the status (orientation, position, and movement) of the corresponding viewing user. This allows the distributor user to distribute while understanding the status of the viewing users.

[0111] In this case, the broadcaster avatar M2 drawn on the broadcaster avatar display layer J0 may be drawn semi-transparently so that the broadcaster user can easily grasp the state of the user avatar display layer J1. In this case, the broadcaster user can easily check the state of the user avatar display layer J1 behind the broadcaster avatar M2. Note that the broadcaster avatar M2 may be switchable between a semi-transparent state and a solid state by input from the broadcaster user. In this case, the broadcaster user can easily selectively check the state of the broadcaster avatar M2 and the state of the user avatar display layer J1 behind the broadcaster avatar M2.

[0112] The background layer J3 may depict the walls of the distribution room 800, etc. Alternatively, the background layer J3 may depict a view beyond the walls of the distribution room 800. Note that the viewer avatar M1 of the viewing user of the specific content related to the distribution image H2 may enter a virtual space (hereinafter referred to as the "viewing space") different from the content production space. In this case, the background related to the viewing space may be depicted in the background layer J3. In this case, the distribution image H21 for the viewing user (see FIG. 13), which is an image of the viewing space, may have a different background layer J3 from the distribution image H20 for the distribution user (see FIG. 12). In this way, there may be inconsistencies between the viewing space and the virtual space (content production space) represented by the distribution image H20 for the distribution user. This allows the distribution image H21 for the viewing user to be optimized from a different perspective than the distribution image H20 for the distribution user.

[0113] In a modified example, as schematically shown in FIG. 8B , the broadcasting room 800 may have a wall 810 made of a half-mirror (one-way mirror) on the front side. In this case, the wall 810 has a reflective function similar to that of the mirror 805, allowing the interior of the room to be seen from the outside but preventing the outside from being seen from the inside. In this case, the broadcasting user can broadcast using the wall 810 in the same manner as the mirror 805 described above. Meanwhile, the viewing user can view the broadcasting user (broadcaster avatar M2) in the broadcasting room 800 through the wall 810. In this case, the background layer J3 of the broadcasting image H2 may include a drawing of a specially constructed stage or the like instead of the interior of the broadcasting room 800.

[0114] Next, with reference to FIG. 15 and subsequent figures, an example of the configuration of the virtual reality generation system 1 described above will be described in order.

[0115] Fig. 15 is a schematic block diagram showing the functions of terminal device 20A on the content distribution side. Fig. 16 is an explanatory diagram of an example of avatar drawing information. In the explanations of Figs. 15 and 16, unless otherwise stated, a distribution user refers to a single distribution user, and a distribution image H2 (H20) refers to a distribution image H2 that forms specific content by that single distribution user.

[0116] As shown in FIG. 15, the terminal device 20A includes a first drawing processing unit 200A, a second drawing processing unit 200B, a first information generation unit 210A, a first communication unit 220A, a first display unit 230A, a first memory unit 240A, and a first user interface unit 250A.

[0117] The functions of the first rendering processing unit 200A, the second rendering processing unit 200B, the first information generation unit 210A, the first communication unit 220A, the first display unit 230A, and the first user interface unit 250A can be realized by the terminal control unit 25 of the terminal device 20A shown in Fig. 1 executing a virtual reality application in cooperation with the terminal communication unit 21, the terminal storage unit 22, the display unit 23, and the input unit 24. Furthermore, the first storage unit 240A can be realized by the terminal storage unit 22 of the terminal device 20A shown in Fig. 1.

[0118] The first rendering processor 200A renders the above-described distribution image H20 (see FIG. 12) (an example of an image of the first virtual space) for the broadcast user. The first rendering processor 200A generates the distribution image H20 (see FIG. 12) for the broadcast user at, for example, a predetermined frame rate.

[0119] The first rendering processor 200A includes a first rendering unit 201A, a second rendering unit 202A, and a third rendering unit 203A.

[0120] The first drawing unit 201A draws an image area (an example of a first image area) of the broadcast avatar display layer J0 in a broadcast image H20 (see FIG. 12) for a broadcast user. The broadcast avatar display layer J0 in the broadcast image H20 is as described above. The first drawing unit 201A may draw a broadcaster avatar M2 in the image area of ​​the broadcast avatar display layer J0 based on, for example, avatar drawing information stored in the first storage unit 240A (see Table 700 in FIG. 16). FIG. 16 shows a schematic diagram of an example of avatar drawing information in Table 700.

[0121] In this case, the avatar drawing information is such that each avatar ID is associated with a facial part ID, a hairstyle part ID, a clothing part ID, etc. Appearance-related part information such as a facial part ID, a hairstyle part ID, and a clothing part ID is a parameter that characterizes an avatar and may be selected by each user. For example, multiple types of appearance-related information such as a facial part ID, a hairstyle part ID, and a clothing part ID for an avatar may be prepared. Furthermore, with regard to the facial part ID, a part ID may be prepared for each type such as face shape, eyes, mouth, and nose, and the information related to the facial part ID may be managed as a combination of IDs of each part that constitutes the face. In this case, each avatar can be drawn not only on the server device 10 but also on the terminal device 20 side based on each appearance-related ID linked to each avatar ID.

[0122] In addition, for example, when the broadcaster avatar M2 opens the closet 801 (see FIG. 8A) and changes clothes (i.e., changes the ID related to hairstyle or clothing), the first drawing unit 201A updates the appearance of the broadcaster avatar M2 accordingly.

[0123] Furthermore, the first drawing unit 201A changes the state of the broadcaster avatar M2 based on the detection result (detection result of the broadcaster user's state) from the first detection unit 211A (described below) of the first information generation unit 210A. In this embodiment, as described above, the first drawing unit 201A draws the broadcaster avatar M2 in a manner that replaces the state of the broadcaster user facing the mirror 805 (see FIG. 8A) with the broadcaster avatar M2.

[0124] For example, if the broadcaster avatar M2 is in the form of a character facing forward, the first drawing unit 201A may link the orientation of the broadcaster avatar M2 to the orientation of the broadcaster user, such that when the broadcaster user faces right, the broadcaster avatar M2 faces left, and when the broadcaster user faces down, the broadcaster avatar M2 faces down. In this case, the orientation may be just the face, just the body, or a combination of these. This increases the consistency (linkage) between the orientations of the broadcaster avatar M2 and the broadcaster user, enabling a wider variety of expressions depending on the orientation of the broadcaster avatar M2.

[0125] Furthermore, if the broadcaster avatar M2 is in the form of a character with a gaze direction, the first drawing unit 201A may link the gaze direction of the broadcaster avatar M2 to the gaze direction of the broadcaster user, such that when the broadcaster user gazes to the right, the gaze of the broadcaster avatar M2 gazes to the left, and when the broadcaster user gazes downward, the gaze of the broadcaster avatar M2 gazes downward. Various eye movements, such as blinking, may also be linked. Movements of the nose, mouth, and the like may also be linked. This increases the consistency (linkage) of each feature between the broadcaster avatar M2 and the broadcaster user, enabling the broadcaster avatar M2 to have a wider variety of facial expressions.

[0126] Furthermore, if the broadcaster avatar M2 is in the form of a character with hands, the first drawing unit 201A may link the movement of the broadcaster avatar M2's hands to the movement of the broadcaster user's hands, such that when the broadcaster user raises their right hand, the broadcaster avatar M2 raises their left hand, and when the broadcaster user raises both hands, the broadcaster avatar M2 raises both hands. The movements of each part of the hand, such as the fingers, may also be linked. Other parts, such as the feet, may also be linked in a similar manner. This increases the consistency (linkage) between the broadcaster avatar M2 and the broadcaster user, enabling a greater variety of expressions through the movements of the broadcaster avatar M2's parts.

[0127] Furthermore, if the position (drawing position) of the broadcaster avatar M2 in the broadcast image H20 is variable (i.e., if the broadcaster avatar M2 is a character whose position changes), the first drawing unit 201A may change the position of the broadcaster avatar M2 in the broadcast image H20 according to the broadcasting user's position (position relative to the mirror 805), such that when the broadcasting user moves to the right, the broadcaster avatar M2 moves to the left, and when the broadcasting user moves away from the mirror 805, the broadcaster avatar M2 moves away. This increases the consistency (linkage) between the movements (positions) of the broadcaster avatar M2 and the broadcasting user, and enables diversification of expressions by changing the position of the broadcaster avatar M2.

[0128] The second drawing unit 202A draws an image area (an example of a second image area) of the interface display layer J2 in the delivery image H20 for the broadcast user (see FIG. 12). The interface display layer J2 in the delivery image H20 is as described above.

[0129] The second drawing unit 202A draws the various operation areas G100 (see FIG. 11) and the small smartphone window area G120 (see FIG. 12) described above. In this case, the second drawing unit 202A may always draw the various operation areas G100 and the small smartphone window area G120, or may omit drawing some or all of them as appropriate. For example, the second drawing unit 202A may output operation areas G117 and G118 (see FIG. 11) for collaboration approval / disapproval in response to a collaboration approval / disapproval instruction from the viewing user information acquisition unit 222A described later. This increases the variety of operations that can be performed via the interface display layer J2 while simplifying (making the display of) the interface display layer J2 (easier to view).

[0130] The third drawing unit 203A draws image areas of the remaining layers (an example of a third image area) in the delivery image H20 for the delivery user (see FIG. 12). That is, the third drawing unit 203A draws image areas of layers other than the delivery avatar display layer J0 and the interface display layer J2 in the delivery image H20. For example, the third drawing unit 203A may draw each image area of ​​the background layer J3 and the auxiliary information display layer J5 according to the hierarchical structure of the delivery image H20.

[0131] In this embodiment, as described above, as an example, the delivery image H20 includes, from the front, a delivery avatar display layer J0, an interface display layer J2, an auxiliary information display layer J5, a user avatar display layer J1, and a background layer J3.

[0132] Therefore, in this embodiment, the third drawing unit 203A draws the image areas of the auxiliary information display layer J5, the user avatar display layer J1, and the background layer J3.

[0133] In response to a gift drawing instruction from the viewing user information acquisition unit 222A (described later), the third drawing unit 203A may draw a gift object in the image area of ​​the auxiliary information display layer J5, for example, based on drawing information of the gift object stored in the first storage unit 240A. Note that the drawing information related to the gift object may be stored in the first storage unit 240A for each gift ID. The gift drawing instruction may include a gift ID that identifies the gift object to be drawn and coordinate information. The coordinate information is information that indicates the drawing position in the distribution image H20 and may change over time. The coordinate information may also include motion information that represents the motion of the gift object. The motion information may represent, for example, a motion such as falling while spinning. This allows for a greater variety in the expression of the gift object.

[0134] Furthermore, in response to a viewer user drawing instruction from the viewer user information acquisition unit 222A (described later), the third drawing unit 203A may draw a viewer avatar M1 (viewer avatar M1 of the viewing user) in the image area of ​​the user avatar display layer J1, for example, based on avatar drawing information (see Table 700 in FIG. 16 ) stored in the first storage unit 240A. The viewer user drawing instruction may include various information for drawing the viewer avatar M1 associated with the viewing user viewing specific content. For example, the various information for drawing the viewer avatar M1 may include information about the viewer avatar M1 (e.g., avatar ID, facial part ID, hairstyle part ID, clothing part ID, etc.) and coordinate information. Furthermore, if the viewer avatar M1 is to be given movement, the viewer user drawing instruction may include information representing the state of the viewing user (or information representing the state of the viewer avatar M1 based thereon). This makes it possible to express the movement of the viewing user in the distribution image H20 for the distribution user, thereby enhancing the sense of unity between the distribution user and the viewing user.

[0135] Furthermore, the third drawing unit 203A may draw various comments in the image area of ​​the user avatar display layer J1 in response to a comment drawing instruction from the viewing user information acquisition unit 222A, which will be described later. The comment drawing instruction may include a comment ID and coordinate information. Note that drawing information related to standard comments may be stored in the first storage unit 240A for each comment ID. Note that instead of the comment ID related to the standard comment, text information (e.g., a message such as a chat) that may be transmitted from the terminal device 20B on the content viewing side may be used. This allows each viewing user to transmit a unique message, etc., thereby enhancing the sense of unity between the broadcasting user and the viewing user in the broadcasting user's distribution image H20.

[0136] Furthermore, the third drawing unit 203A may draw a background image corresponding to the background ID selected by the broadcast user on the background layer J3. The background image may be stored in the first storage unit 240A for each background ID. Note that the background image may be customized by the broadcast user.

[0137] The second rendering processor 200B renders the above-described home image H1 (see FIG. 6) (an example of an image of the second virtual space). The second rendering processor 200B generates the home image H1 (see FIG. 6) at a predetermined frame rate, for example.

[0138] The second rendering processor 200B includes a fourth rendering unit 204B and a fifth rendering unit 205B.

[0139] The fourth drawing unit 204B draws an image area (an example of a fourth image area) of the interface display layer J2 in the home image H1 (see FIG. 6). The interface display layer J2 in the home image H1 is as described above. The fourth drawing unit 204B may change the arrangement of the multiple operation areas G300 in response to a command from the first input processing unit 252A (described later) (see FIGS. 6 and 7A).

[0140] The fifth drawing unit 205B draws image areas (an example of a fifth image area) of the remaining layers in the home image H1 (see FIG. 6). That is, the fifth drawing unit 205B draws image areas of the layers other than the interface display layer J2 in the home image H1. For example, the fifth drawing unit 205B may draw each image area of ​​the background layer J3 and the auxiliary information display layer J5 according to the hierarchical structure of the home image H1. This makes it possible to efficiently provide various information that may be necessary to the user without significantly increasing the processing load related to drawing.

[0141] In the present embodiment, as described above, as an example, the home image H1 includes, from the front side, an interface display layer J2, an auxiliary information display layer J5, and a background layer J3.

[0142] Therefore, in this embodiment, the fifth drawing unit 205B draws the image areas of the auxiliary information display layer J5 and the background layer J3.

[0143] The first information generation unit 210A generates various information related to the broadcast user (hereinafter referred to as "broadcast user information"). The broadcast user information includes information indicating the broadcast user's state (the broadcast user's state detected by the first detection unit 211A) used by the first user interface unit 250A described below. The broadcast user information also includes information necessary to draw a broadcast image H21 (see FIG. 13) for the viewing user. For example, the broadcast user information includes information for drawing the broadcaster avatar M2 (e.g., avatar ID, facial part ID, hairstyle part ID, clothing part ID, etc.). Furthermore, if the specific content by the broadcast user includes audio from the broadcast user, the broadcast user information may include audio information acquired via a microphone or the like of the input unit 24. This allows the various pieces of information constituting the specific content to be handled separately, enabling distributed processing.

[0144] In this embodiment, the first information generation unit 210A includes a first detection unit 211A, a first audio information generation unit 212A, and a first text information generation unit 213A.

[0145] The first detection unit 211A detects the broadcast user's state (the user's orientation, position, movement, or the like). The first detection unit 211A may detect the broadcast user's state via the input unit 24 described above. When the home image H1 is being drawn, the first detection unit 211A may detect the broadcast user's state, for example, at a cycle corresponding to the frame rate of the home image H1. Furthermore, when the broadcast user's broadcast image H20 is being drawn, the first detection unit 211A may detect the broadcast user's state, for example, at a cycle corresponding to the frame rate of the broadcast user's broadcast image H20 (see FIG. 12). This allows the broadcast user's state (and the associated display of the broadcaster avatar M2) to be detected (updated) frequently.

[0146] The first detection unit 211A may switch between an ON / OFF state in response to, for example, an operation by the broadcasting user. For example, when the broadcasting user operates a button for starting broadcasting (see operation area G107 in FIG. 11 ) that may be displayed on the interface display layer J2, the operation may be detected by the first input detection unit 251A, which will be described later, and the first detection unit 211A may transition to the ON state in response to a command from the first input processing unit 252A, which will be described later. This increases the variety of operations that can be performed via the interface display layer J2 while simplifying the display of the interface display layer J2 (making it easier to see). By simplifying the display of the interface display layer J2, the visibility of the layers behind it may be improved.

[0147] The first voice information generation unit 212A generates voice information based on the broadcast user's speech. For example, the first voice information generation unit 212A generates the broadcast user's voice information via a microphone or the like of the input unit 24. The first voice information generation unit 212A may generate the broadcast user's voice information by processing voice data obtained via the input unit 24. In this case, the processing method may be selectable by the broadcast user, or the voice data processing may be performed manually by the broadcast user.

[0148] The first audio information generation unit 212A may switch between an on / off state in response to, for example, an operation by the broadcasting user. For example, when the broadcasting user operates a mute button (not shown) that may be displayed on the interface display layer J2, the operation may be detected by a first input detection unit 251A (described later), and the first audio information generation unit 212A may transition to an off state in response to a command from a first input processing unit 252A (described later).

[0149] First text information generation unit 213A generates text information based on the broadcast user's speech and / or character input. For example, first text information generation unit 213A generates text information when the broadcast user interacts (chat, etc.) with the viewing user.

[0150] The first text information generation unit 213A may be switched between an ON / OFF state in response to, for example, an operation by the broadcasting user. For example, when the broadcasting user operates a comment button (see operation area G101 in FIG. 11 ) that may be displayed on the interface display layer J2, the operation may be detected by the first input detection unit 251A, which will be described later, and the first text information generation unit 213A may transition to the ON state in response to a command from the first input processing unit 252A, which will be described later. This increases the variety of operations that can be performed via the interface display layer J2 while simplifying (making the display of) the interface display layer J2 (making it easier to see).

[0151] In this case, the first information generation unit 210A may include, in the broadcast user information, a set of information representing the broadcast user's status (or information representing the status of the broadcaster avatar M2 based on that status) (hereinafter simply referred to as "status information of the broadcaster avatar M2"), information about the broadcaster avatar M2 (e.g., avatar ID, facial part ID, hairstyle part ID, clothing part ID, etc.), and, as appropriate, audio information and text information. In this case, the first information generation unit 210A may generate the broadcast user information in a manner that adds a timestamp to dynamically changing information such as the status information, audio information, and text information of the broadcaster avatar M2. As a result, upon receiving such broadcast user information, the terminal device 20B on the content viewing side can output, based on the received broadcast user information, the above-mentioned broadcast image H21 accompanied by audio output, as specific content by the broadcast user.

[0152] The first communication unit 220A communicates with the server device 10 and other terminal devices 20 (for example, the terminal device 20B on the content viewing side). In this embodiment, the first communication unit 220A includes a distribution processing unit 221A and a viewing user information acquisition unit 222A.

[0153] The distribution processing unit 221A transmits the distribution user information generated by the first information generation unit 210A to the terminal device 20B on the content viewing side. In a modified example, the distribution processing unit 221A may transmit video data based on the distribution image H20 generated by the first rendering processing unit 200A to the terminal device 20B.

[0154] Distribution processing unit 221A may transmit distributor user information to terminal device 20B on the content viewing side in real time (i.e., realizing live distribution) in response to, for example, an operation by the distributor user, or may transmit distributor user information to terminal device 20B on the content viewing side in non-real time. In the non-real time case, distribution processing unit 221A may transmit video data after manual editing by the distributor user or edited distributor user information to terminal device 20B on the content viewing side.

[0155] The viewing user information acquisition unit 222A acquires various types of viewing user information from the terminal device 20B on the content viewing side. Based on the acquired viewing user information, the viewing user information acquisition unit 222A generates the above-mentioned gift drawing instruction, viewing user drawing instruction, comment drawing instruction, collaboration approval / disapproval instruction, etc. Note that the viewing user information includes information necessary for generating these gift drawing instructions, viewing user drawing instructions, comment drawing instructions, collaboration approval / disapproval instructions, etc.

[0156] The first display unit 230A outputs a distribution image H20 (see FIG. 12) for the distribution user generated by the first drawing processing unit 200A and a home image H1 generated by the second drawing processing unit 200B to the display unit 23 of the terminal device 20A. Note that in this embodiment, as described above, the display unit 23 is in the form of a head-mounted display, for example.

[0157] The first storage unit 240A stores the above-mentioned avatar drawing information (see table 700 in FIG. 16) and the like.

[0158] The first user interface unit 250A detects various inputs from the broadcast user via the input unit 24 and executes processing in accordance with the various inputs. The first user interface unit 250A includes a first input detection unit 251A and a first input processing unit 252A.

[0159] The first input detection unit 251A detects various inputs by the broadcast user via the interface display layer J2 described above. For example, the first input detection unit 251A detects inputs via the various operation areas G100 (see FIG. 11), the smartphone small window area G120 (see FIG. 12), the operation area G300, etc.

[0160] In this case, the first input detection unit 251A may detect input by the broadcasting user via the interface display layer J2 based on the state of the broadcasting user detected by the first detection unit 211A and the state of the interface display layer J2 (for example, the position and display state of the various operation areas G100 and the small smartphone window area G120). For example, when the first input detection unit 251A detects a tap action on one of the various operation areas G100, it may detect a selection operation on that one operation area. This allows the user to perform a selection operation with a simple movement in the virtual space.

[0161] The first input processing unit 252A executes various processes (examples of various second processes) in response to various inputs detected by the first input detection unit 251A on the home image H1. The various processes are arbitrary, and as described above, may be, for example, a process for changing the layout of a user interface (e.g., a plurality of planar operation areas G300) or a process for moving a user to any location or a specific location in a virtual space (e.g., a process for moving a broadcasting user to a broadcasting location). Note that the various processes may involve various drawing processes or screen transitions (e.g., movement in a virtual space) by the second drawing processing unit 200B as appropriate.

[0162] Additionally, the first input processing unit 252A executes various processes (examples of various first processes) in response to various inputs detected by the first input detection unit 251A in the distribution image H20 for the broadcast user (see FIG. 12). The various processes are arbitrary, and as described above, may be, for example, a process for starting distribution of specific content, a process for ending distribution of specific content, a voice input process, a process for the broadcast user to receive a gift, etc. Note that the various processes may appropriately involve various drawing processes or screen transitions (e.g., movement in virtual space) by the first drawing processing unit 200A. For example, a process for transitioning to a state in which a process for receiving a gift can be executed may involve the drawing of notification information (notification information that a gift has been sent) by the third drawing unit 203A of the first drawing processing unit 200A, or the drawing of a receiving button by the second drawing unit 202A of the first drawing processing unit 200A.

[0163] As described above, according to the present embodiment, the delivery image H20 for the broadcast user has multiple layers, and the processing load can be efficiently reduced by layering the image of the delivery image H20 for the broadcast user. For example, when delivering specific content in video format, the image area of ​​the delivery avatar display layer J0 and the image area of ​​the interface display layer J2 can be drawn (updated) more frequently than the image area of ​​the interface display layer J2, thereby efficiently reducing the overall processing load when drawing the delivery image H20 for the broadcast user. Furthermore, by layering the delivery image H20 for the broadcast user, a sense of distance as a "space" can be effectively given to the user. Furthermore, by layering the delivery image H20 for the broadcast user, it is possible to arrange the image areas of each layer at an appropriate distance for each attribute, for example. This makes it possible to, for example, simultaneously draw a user interface with high operability that is less likely to cause motion sickness and draw a broadcaster avatar M2 that is easy to see.

[0164] FIG. 17 is a schematic block diagram showing the functions of the terminal device 20B on the content viewing side.

[0165] As shown in FIG. 17, the terminal device 20B includes a first drawing processing unit 200A′, a second drawing processing unit 200B, a second information generation unit 210B, a second communication unit 220B, a second display unit 230B, a second memory unit 240B, and a second user interface unit 250B.

[0166] The functions of the first rendering processing unit 200A', the second rendering processing unit 200B, the second information generation unit 210B, the second communication unit 220B, the second display unit 230B, and the second user interface unit 250B can be realized by the terminal control unit 25 of the terminal device 20B shown in Fig. 1 executing a virtual reality application in cooperation with the terminal communication unit 21, the terminal storage unit 22, the display unit 23, and the input unit 24. In addition, the second storage unit 240B can be realized by the terminal storage unit 22 of the terminal device 20B shown in Fig. 1.

[0167] The first rendering processor 200A' renders a delivery image H21 (see FIG. 13) for the viewing user in response to a broadcast user drawing instruction from the broadcast user information acquisition unit 222B, which will be described later. In this case, the broadcast user drawing instruction is generated based on broadcast user information transmitted from the terminal device 20A on the content distribution side (broadcast user information generated by the first information generation unit 210A of the terminal device 20A). The first rendering processor 200A' may generate the delivery image H21 (see FIG. 13) for the viewing user at a frame rate corresponding to the update cycle of the broadcast user information, for example.

[0168] The first rendering processor 200A' includes a first rendering unit 201B, a second rendering unit 202B, and a third rendering unit 203B.

[0169] The first drawing unit 201B draws an image area (an example of a first image area) of the distribution avatar display layer J0 in the distribution image H21 for the viewing user (see FIG. 13). The distribution avatar display layer J0 in the distribution image H21 for the viewing user is as described above. The first drawing unit 201B may draw a broadcaster avatar M2 in the image area of ​​the distribution avatar display layer J0 based on the broadcast user information and, for example, avatar drawing information stored in the first storage unit 240A (see Table 700 in FIG. 16). In this case, because the avatar drawing information stored in the first storage unit 240A is used, the broadcaster avatar M2 can be drawn with processing that imposes a relatively low load.

[0170] The operation of the first drawing unit 201B has the same function as the first drawing unit 201A of the terminal device 20A on the content distribution side described above. In this case, various information for drawing the broadcaster avatar M2 may be included in the broadcaster user drawing instruction. That is, the broadcaster user drawing instruction may include information about the broadcaster avatar M2 (e.g., avatar ID, face part ID, hairstyle part ID, clothing part ID, etc.), information representing the broadcaster user's state (or information representing the state of the broadcaster avatar M2 based thereon), etc.

[0171] The second drawing unit 202B draws an image area (an example of a second image area) of the interface display layer J2 in the delivery image H21 for the viewing user (see FIG. 13). The operation of the second drawing unit 202B has the same function as the second drawing unit 202A of the terminal device 20A on the content delivery side described above. However, an operation area different from the interface display layer J2 in the delivery image H20 for the broadcast user (see FIG. 12) may be drawn in the interface display layer J2 in the delivery image H21 for the viewing user. For example, as shown in FIG. 13, the second drawing unit 202B may draw an operation area G108 for sending a gift to the broadcast user, an operation area G109 for sending a request for a collaborative broadcast to the broadcast user, etc. Note that a request for a collaborative broadcast may be issued by the broadcast user, in which case an operation area for sending a request for a collaborative broadcast to the viewing user may be drawn in the delivery image H20 for the broadcast user (see FIG. 12). This promotes collaborative broadcasting and effectively encourages interaction between users.

[0172] The third drawing unit 203B draws the image area of ​​the remaining layer (an example of the third image area) in the delivery image H21 for the viewing user (see FIG. 13). The operation of the third drawing unit 203B has the same function as the third drawing unit 203A of the terminal device 20A on the content delivery side described above. However, the drawing content in the auxiliary information display layer J5 may be different as appropriate. For example, in the delivery image H21 for the viewing user, as shown in FIG. 13, the viewer avatar M1 (see FIG. 12) may not be drawn.

[0173] The second rendering processor 200B renders the above-described home image H1 (see FIG. 6) (an example of an image of the second virtual space). The second rendering processor 200B generates the home image H1 (see FIG. 6) at a predetermined frame rate, for example.

[0174] The second rendering processor 200B includes a fourth rendering processor 204B and a fifth rendering processor 205B. The second rendering processor 200B may be substantially similar to the second rendering processor 200B of the terminal device 20A. This allows for an efficient configuration that utilizes common hardware resources.

[0175] The second information generation unit 210B generates various types of viewing user information related to the viewing user. The various types of viewing user information include information indicating the state of the viewing user (the state of the viewing user detected by the second detection unit 211B) used by the second user interface unit 250B described later. The various types of viewing user information also include information necessary for the viewing user information acquisition unit 222A of the terminal device 20A on the content distribution side described above to generate a gift drawing instruction, a viewing user drawing instruction, a comment drawing instruction, a collaboration approval / disapproval instruction, etc.

[0176] In this embodiment, the second information generation unit 210B includes a second detection unit 211B, a second audio information generation unit 212B, and a second text information generation unit 213B.

[0177] The second detection unit 211B detects the state of the viewing user (the user's orientation, position, movement, or the like). The second detection unit 211B may detect the state of the viewing user via the input unit 24 described above.

[0178] When the home image H1 is being drawn, the second detection unit 211B may detect the state of the viewing user, for example, at a cycle corresponding to the frame rate of the home image H1. Furthermore, when a delivery image H21 (see FIG. 13) for the viewing user is being drawn, the second detection unit 211B may detect the state of the viewing user, for example, at a cycle corresponding to the update cycle of the delivery user information.

[0179] The second voice information generation unit 212B generates voice information based on the speech of the viewing user. For example, the second voice information generation unit 212B generates the voice information of the viewing user via a microphone or the like of the input unit 24. Note that the second voice information generation unit 212B may generate the voice information of the viewing user by processing voice data obtained via the input unit 24. In this case, the processing method may be selectable by the viewing user, or the processing of the voice data may be manually realized by the viewing user.

[0180] The second audio information generation unit 212B may be switched between an ON / OFF state in response to, for example, an operation by a viewing user. For example, when a viewing user operates a mute button (not shown) that may be displayed on the interface display layer J2 in a distribution image H21 for the viewing user (see FIG. 13), the operation may be detected by a second input detection unit 251B (described later), and the second audio information generation unit 212B may transition to an OFF state in response to a command from a second input processing unit 252B (described later).

[0181] Second text information generation unit 213B generates text information based on the speech and / or character input of the viewing user. For example, second text information generation unit 213B generates text information when the viewing user has a conversation (chat, etc.) with the broadcasting user.

[0182] The second text information generation unit 213B may be switched between an ON / OFF state in response to, for example, an operation by a viewing user. For example, when a viewing user operates a comment button (not shown in FIG. 13) that may be displayed on the interface display layer J2, the operation may be detected by a second input detection unit 251B (described later), and the second text information generation unit 213B may transition to an ON state in response to a command from a second input processing unit 252B (described later). This increases the variety of operations that can be performed via the interface display layer J2 while simplifying (making the display of) the interface display layer J2.

[0183] In this case, the second information generator 210B may include, in the viewer user information, a set of information representing the state of the viewer user (or information representing the state of the viewer avatar M1 based on the state information) (hereinafter simply referred to as "state information of the viewer avatar M1"), information about the viewer avatar M1 (e.g., avatar ID, facial part ID, hairstyle part ID, clothing part ID, etc.), and, as appropriate, audio information. In this case, the second information generator 210B may generate the viewer user information in a manner that adds a timestamp to dynamically changing information such as the state information and audio information of the viewer avatar M1. As a result, upon receiving such viewer user information, the terminal device 20A on the content distribution side can render the viewer avatar M1 in the user avatar display layer J1 in the distribution image H20 as described above, based on the received viewer user information.

[0184] The second communication unit 220B communicates with the server device 10 and other terminal devices 20 (for example, the terminal device 20A on the content distribution side). In this embodiment, the second communication unit 220B includes a distribution processing unit 221B and a distribution user information acquisition unit 222B.

[0185] The distribution processing unit 221B transmits the viewing user information generated by the second information generating unit 210B to the terminal device 20A on the content distribution side.

[0186] The broadcast user information acquisition unit 222B acquires broadcast user information from the terminal device 20A on the content distribution side, and provides the above-mentioned broadcast user drawing instruction to the first drawing processing unit 200A'. The broadcast user information is generated by the distribution processing unit 221A of the terminal device 20A on the content distribution side, as described above.

[0187] The second display unit 230B outputs the delivery image H20 and the home image H1 generated by the first rendering processing unit 200A′ and the second rendering processing unit 200B to the display unit 23 of the terminal device 20B. Note that in this embodiment, as described above, the display unit 23 is in the form of a head-mounted display, for example.

[0188] The second storage unit 240B stores the above-mentioned avatar drawing information (see table 700 in FIG. 16) and the like.

[0189] The second user interface unit 250B detects various inputs from the viewing user via the input unit 24 and executes processing in accordance with the various inputs. The second user interface unit 250B includes a second input detection unit 251B and a second input processing unit 252B.

[0190] The second input detection unit 251B detects various inputs by the viewing user via the interface display layer J2 described above. For example, the second input detection unit 251B detects inputs via the interface display layer J2 drawn by the second drawing unit 202B or the fourth drawing unit 204B described above.

[0191] In this case, the second input detection unit 251B may detect an input by the viewing user via the interface display layer J2 based on the state of the viewing user detected by the second detection unit 211B and the state of the interface display layer J2 (e.g., the positions and display states of various operation areas G100 and the small smartphone window area G120). For example, the second input detection unit 251B may detect various operations by the viewing user on the above-mentioned multiple planar operation areas G300 in the home image H1, or detect various operations by the viewing user on various operation areas (such as an operation area for sending a request for collaborative distribution to the distribution user) in the distribution image H21 for the viewing user (see FIG. 13).

[0192] The second input processing unit 252B executes various processes (examples of various second processes) in response to various inputs detected by the second input detection unit 251B in the home image H1. The various processes are arbitrary, and as described above, may be, for example, a process for changing the layout of a user interface (e.g., multiple planar operation areas G300) or a process for moving a user to any location or a specific location in the virtual space (e.g., a process for moving a viewing user to a viewing location). The various processes may also involve various drawing processes or screen transitions (e.g., movement in the virtual space) by the second rendering processing unit 200B, as appropriate. For example, a process for transitioning to a state in which a gift-giving process can be executed may involve the fourth rendering unit 204B of the second rendering processing unit 200B drawing a gift selection screen, etc.

[0193] Additionally, the second input processing unit 252B executes various processes (examples of various second processes) in response to various inputs detected by the second input detection unit 251B in the delivery image H21 for the viewing user (see FIG. 13). The various processes are arbitrary, and as described above, may be, for example, a process for ending viewing of specific content, a voice input process, a process for sending a gift to the broadcasting user, etc. Note that the various processes may appropriately involve various drawing processes and screen transitions (e.g., movement in virtual space) by the first drawing processing unit 200A'. For example, the process of sending a gift to the broadcasting user may involve the drawing of notification information (notification information that the gift has been received) by the third drawing unit 203B of the first drawing processing unit 200A'.

[0194] FIG. 18 is a schematic block diagram showing the functions of the server device 10. As shown in FIG.

[0195] The server device 10 mainly includes a communication unit 100 and a storage unit 110.

[0196] The communication unit 100 can communicate various information (e.g., distribution user information, viewing user information, etc.) required for the distribution and / or viewing of specific content by the distribution user between the terminal device 20A on the content distribution side and / or the terminal device 20B on the content viewing side.

[0197] The storage unit 110 can store various information required for the distribution and / or viewing of specific content by the distribution user.

[0198] As described above, according to the present embodiment, the delivery image H21 for the viewing user has multiple layers, and the processing load can be efficiently reduced by layering the image of the delivery image H21 for the viewing user. For example, the image area of ​​the interface display layer J2 that is in front of the user's viewpoint can be rendered (updated) more frequently than other image areas, thereby efficiently reducing the overall processing load when rendering the delivery image H21 for the viewing user. Furthermore, layering the delivery image H21 for the viewing user can effectively give the user a sense of distance in a "space." Furthermore, layering the delivery image H21 for the viewing user can, for example, arrange the image areas of each layer at an appropriate distance according to their attributes. This makes it possible to, for example, render a user interface with high operability that is less likely to cause motion sickness, while also rendering an operation area G300 or the like that is easy to see.

[0199] In this embodiment, the server device 10 does not have the drawing function of the home image H1 or the delivery image H2 described above, and such drawing function is realized by the terminal device 20A on the content delivery side and / or the terminal device 20B on the content viewing side, as described above. However, part or all of the drawing function of the home image H1 and / or the delivery image H2 may be realized by the server device 10. This may also be the case for functions other than the drawing function, as appropriate.

[0200] Next, an example of the operation performed in the virtual reality generation system 1 shown in FIG. 1 will be described with reference to FIG. 19 and subsequent figures.

[0201] FIG. 19 is a flow diagram showing an example of the operation of terminal device 20A on the content distribution side performed in virtual reality generation system 1 shown in FIG. 1, up to the start of distribution of specific content.

[0202] First, in step S190, the user who will become a broadcasting user wears terminal device 20A, which is a head-mounted display, and launches a virtual reality application. This positions the user within the home space (step S191) and allows them to view a home image H1. The user operates the operation area of ​​interface display layer J2 in the home image H1 to move to the content production space (step S192). After moving to the content production space, the broadcasting user prepares by, for example, changing the broadcaster avatar M2 (step S193), and then stands in front of mirror 805 and views a broadcasting image H20 (pre-broadcast preparation image) (step S194). Note that the broadcasting image H20 as a pre-broadcast preparation image may include only the broadcasting avatar display layer J0 and the interface display layer J2, or may include only the broadcasting avatar display layer J0, the interface display layer J2, and the background layer J3. In addition, in the distribution image H20, which is a preparatory image before distribution, an operation area G107 (see FIG. 11) for starting distribution may be drawn on the interface display layer J2. In another embodiment, distribution may start simultaneously with moving to the content production space.

[0203] When the broadcasting user sees the broadcast image H20 and decides to start broadcasting, the broadcasting user operates the corresponding operation area G107 (see FIG. 11) in the interface display layer J2 to start broadcasting of the specific content including the broadcast image H20 (step S195). When the broadcasting of the specific content starts in this way, an operation area G300 (see FIGS. 6 and 7A, etc.) in which the specific content can be selected is rendered in the interface display layer J2 in the home image H1 of the terminal device 20B on the content viewing side.

[0204] FIG. 20 is a flow diagram showing an example of the operation of terminal device 20B on the content viewing side performed in virtual reality generation system 1 shown in FIG. 1, up to the start of viewing of specific content.

[0205] First, in step S200, a user who will become a viewing user wears the terminal device 20B, which is a head-mounted display, and starts a virtual reality application. This positions the user within the home space (step S201) and allows the user to view the home image H1. The user operates the operation area G300 of the interface display layer J2 in the home image H1 to select desired specific content (step S202). In this way, the viewing user can begin viewing the specific content (step S203).

[0206] Figure 21 is a flow diagram showing an example of the operation of the terminal device 20A on the content distribution side, the terminal device 20B on the content viewing side, and the server device 10 performed in the virtual reality generation system 1 shown in Figure 1 while a specific content is being distributed by a distribution user (i.e., while a viewing user is viewing the specific content).

[0207] In Figure 21, the left side shows the operations performed by terminal device 20A on one content distribution side, the center shows the operations performed by server device 10 (here, one server device 10), and the right side shows the operations performed by terminal device 20B on one content viewing side.

[0208] In step S210, the broadcast user performs an act or the like in front of mirror 805. As a result, terminal device 20A on the content broadcasting side generates broadcast user information corresponding to the act or the like. Terminal device 20A on the content broadcasting side transmits such broadcast user information to server device 10. Note that the broadcast user information may be multiplexed with other information by any multiplexing method and transmitted to server device 10, as long as the condition is met that the correspondence between the information to be transmitted (transmitted) and the timestamp based on the reference time is clear to both terminal device 20A on the content broadcasting side and terminal device 20B on the content viewing side. If such a condition is met, upon receiving the broadcast user information, terminal device 20B on the content viewing side can process the information appropriately according to the timestamp corresponding to the broadcast user information. Regarding the multiplexing method, the broadcast user information may be transmitted via separate channels, or some of the broadcast user information may be transmitted via the same channel. The channel may include a time slot, a frequency band, and / or a spreading code, etc. The method of distributing a moving image (specific content) using a reference time may be realized in the manner disclosed in Japanese Patent No. 6803485, which is incorporated herein by reference.

[0209] Next, in parallel with the operation in step S210, the terminal device 20A on the content distribution side continuously transmits distribution user information for drawing a distribution image H21 for the viewing user to the terminal device 20B on the content viewing side in step S212, and outputs a distribution image H20 for the distribution user (see Figure 12) to the terminal device 20A on the content distribution side.

[0210] The terminal device 20A on the content distribution side can perform the operations in steps S210 and S212 in parallel with the operations in steps S214 to S222 described below.

[0211] Next, in step S214, server device 10 transmits (transfers) the distribution user information that is continuously transmitted from terminal device 20A on the content distribution side to terminal device 20B on the content viewing side.

[0212] In step S216, terminal device 20B on the content viewing side receives the broadcast user information from server device 10 and stores it in storage unit 140. In one embodiment, taking into consideration the possibility that audio information may have a larger capacity than other information and / or the possibility that a failure may occur on the communication line, terminal device 20B on the content viewing side can temporarily store (buffer) the broadcast user information received from server device 10 in terminal storage unit 22 (see FIG. 1).

[0213] In parallel with receiving and storing such distribution user information, in step S218, the terminal device 20B on the content viewing side uses the distribution user information received and stored from the terminal device 20A on the content distribution side via the server device 10 to generate a distribution image H21 for the viewing user and play back the specific content.

[0214] In parallel with the operations in steps S216 and S218 described above, in step S220, terminal device 20B on the content viewing side generates viewing user information and transmits the viewing user information to terminal device 20A on the content distribution side via server device 10. The viewing user information may be generated, for example, only when the viewing user performs an operation to give a gift.

[0215] In step S222, the server device 10 transmits (transfers) the viewing user information received from the terminal device 20B on the content viewing side to the terminal device 20A on the content distribution side.

[0216] In step S224, the terminal device 20A on the content distribution side can receive the viewing user information via the server device .

[0217] In step S226, the terminal device 20A on the content distribution side can basically perform the same operation as in step S210. For example, the terminal device 20A on the content distribution side generates a gift drawing instruction based on the viewing user information received in step S224, and draws a corresponding gift object on the auxiliary information display layer J5 in the distribution image H20.

[0218] In this way, the process shown in FIG. 21 may be continuously executed until the distribution of the specific content by the distribution user is completed or until there are no more viewing users of the specific content.

[0219] 21, the entity that executes each process can be changed in various ways, as described above. For example, among the processes in step S212, the process of generating a delivery image H20 for the broadcast user may be implemented by server device 10 instead of terminal device 20A. Furthermore, among the processes in step S218, the process of generating a delivery image H21 for the viewing user may be implemented by terminal device 20A or server device 10. In this case, in step S216, data for the delivery image H21 for the viewing user may be received instead of the broadcast user information. Furthermore, among the processes in step S226, the process of drawing a gift object on the auxiliary information display layer J5 in the delivery image H20 based on the viewing user information may be implemented by server device 10 instead of terminal device 20A.

[0220] (Variation) The above-described embodiment has been described as using wearable devices (head-mounted displays) as the terminal devices 20A and 20B. However, the virtual reality generation system 1 according to the present invention may also be configured as a metaverse (a virtual space on the Internet) using a mobile phone, smartphone, tablet device, or PC (personal computer) as the terminal device 20A (an example of a first terminal device) and the terminal device 20B (an example of a second terminal device). Below, we will describe variations of this configuration. Regarding the metaverse configuration, we will discuss possible scenes, screen switching, and user interface (UI) aspects. For matters not mentioned below, the description of the above-described embodiment applies within the scope of technical rationality. In the following, the user interface is realized by the input unit 24, the first user interface unit 250A, and / or the second user interface unit 250B in the above-described embodiment.

[0221] First, we will explain the scenes that can be set when configured as a metaverse, and the user interface in each scene.

[0222] In situations where users are viewing together (hereinafter referred to as "shared viewing"), it is preferable that the user interface presents a hierarchical menu of operations such as play, stop, and language settings. In other words, even when using a single large screen for shared viewing, it is preferable that volume, subtitle language settings, play, stop, fast forward, etc. can be operated individually, and that in addition to direct operations, permission settings for these operations are hierarchical. Examples of user interfaces include [Play video > List view > Shared viewing / Only me > Pin here / Operate with everyone] and [Screen operations during playback > (if authorized) Play / Stop / Fast forward / Rewind / End].

[0223] In scenes where emotes (emotional expressions) are given to avatars, since each user looks toward the screen during shared viewing, it is preferable to set them more simply and quickly (for example, semi-automatically) while clearly indicating the target direction, rather than selecting the usual facial expressions and emotes. Examples of the user interface include [Shared screen > Spectator mode > Up, down, left, and right of the joystick change to {smile mark, heart mark, applause mark, semi-auto}] and [Semi-automatically > Follow other people's reactions / automatically react when there are cheers on the screen].

[0224] In presentation mode, where a user is giving a presentation as a speaker, the speaker can see the scenario, the materials to be shared on the screen, and the display user interface, while the other users can hide the menu and user interface (although Virtual Cast can be displayed). An example of the user interface could be something like [Go to next page / Currently speaking text / Next image]. Since the speaker cannot look back, the operations, previews, and prepared images and text are displayed on the screen.

[0225] In a seated answer mode, where users quickly answer questions using only options, such as for quiz shows or online education, an example of a user interface may be one in which each user is presented with a choice of answers and a raise-hand button. To avoid being too flexible, once specified by the organizer, the user interface is fixed until certain conditions are met. The interface consists of a set of questions and answer options, and may include a timer, score, number of coins to bet, etc.

[0226] In the placement mode, a scene in which a building is placed, the hierarchy is [Place → Shop → Outdoor → Purchase (balance, payment, loan) → Confirm → Location → Orientation → Other users can / cannot operate...]. However, if this is directly placed in 3D space, it is difficult to align the orientation because only the ability to "pick up and drop" the object is possible. Furthermore, it is difficult to set subsequent attributes such as "whether others can see, touch, or move around." Therefore, as an example of a user interface, a layout such as [Walk freely in 3D space → Open menu → Shop] can be used. In this case, the 3D space is displayed faintly in the background of the 2D shop screen, so the menu and shop can be hidden from others. While this is displayed, others see a magnifying glass floating above their head, appearing frozen in place to indicate that the "shop" is in operation. When placing an object that involves a collision with others, the "proposed construction site" is displayed to others, and if other users are present, processing is suspended. If the problem is not resolved after waiting for a certain period of time, you can return to the confirmation screen again, and if you cancel, you can go back one step in terms of placement location and orientation.

[0227] Next, we will explain screen switching and the user interface when configured as a metaverse.

[0228] Regarding the user's viewpoint, operating only from the user's viewpoint can be problematic when viewing from a bird's-eye view or a third-person perspective. Therefore, to switch the viewpoint, a bird's-eye view can be obtained by selecting [Bird's-eye View] in the user interface. An example of the user interface may be [Menu → Drone Photo → Confirm to enter drone operation mode]. In this case, the avatar may be fixed, and aerial photography from the drone operated with the stick may be performed by creating motions such as emotes or hands.

[0229] Regarding the layout view when taking a screenshot, in conjunction with the continuous shooting mode, a confirmation screen asking "Do you want to share?" is displayed, and an example of the user interface may be, for example, [open film roll → line up several past shots and select left and right / share top and bottom / filter / layout (portrait, landscape, square) / delete, etc.].

[0230] The game view during a game in the metaverse (In-Space-Game) displays the results and score sheet according to the rules of the game. Information to assist with the game (for example, the target pins in bowling, or the finger placements of the opponent in rock-paper-scissors) may also be displayed. On the other hand, depending on the nature of the game, information such as weapon switching, opponent HP display, and communication with allied groups may not be displayed because it is information that users do not want other users to see.

[0231] If the game involves vehicle operation, such as a car racing game, a ranking display, map display, etc. may be displayed. In the case of a car racing game, control is replaced with steering wheel operation, and when you select to get in a car, the user interface is restricted to a driving UI. In this mode, the hand controller is not a substitute for the hands, but is a substitute for steering wheel and accelerator / brake operation, with left and right movements being used to handle the steering wheel, and up and down movements being used to change gears and operate the accelerator / brake.

[0232] In addition to the above, a view related to smartphone compatibility may also be displayed. This differs from the so-called virtual smartphone display in that it is related to smartphone purchasing decisions and makes it easier to make decisions by displaying a user interface (smartphone UI) on the screen in a way that hides the 3D background. The user interface may be defined to suit the user's environment, for example, regarding how to use the trackpad and stick implemented in various hand controllers (swipe, touch, long press, drag, etc.). Note that the hand controllers, trackpads, and sticks referred to here, including the hand controllers in the car racing game described above, may be real or virtual, rendered in a virtual space.

[0233] Although an embodiment of the present invention has been described above in detail with reference to the drawings, the specific configuration is not limited to this embodiment, and includes designs within the scope of the gist of the present invention. [Explanation of symbols]

[0234] 1 Virtual reality generation system 3 Network 10 Server device 11 Server Communication Unit 12 Server storage unit 13 Server control unit 20, 20A, 20B terminal equipment 21 Terminal communication unit 22 Terminal memory section 23 Display section 24 Input section 25 Terminal control unit 30A, 30B Studio Unit 140 Storage section 200 Communications Department 200A, 200A' First drawing processing unit 201A, 201B 1st drawing section 202A, 202B 2nd drawing section 203A, 203B 3rd drawing section 200B Second drawing processing unit 204B 4th drawing section 205B 5th drawing section 210 Storage section 210A 1st information generation section 210B 2nd information generation section 211A First detection unit 211B Second detection unit 212A first voice information generation unit 212B second audio information generation unit 213A First text information generation unit 213B Second text information generation unit 220A First Communications Department 220B 2nd Communication Department 221A, 221B distribution processing unit 222A Viewing user information acquisition unit 222B Distribution user information acquisition unit 230A 1st display section 230B 2nd display section 240A 1st memory section 240B 2nd storage section 250A 1st User Interface Section 250B Second User Interface Unit 251A 1st input detection unit 251B Second input detection unit 252A First input processing section 252B second input processing section

Claims

1. a drawing processing unit that draws an image of a virtual space having a plurality of layers; The image of the virtual space includes a first image for a first user who delivers content to the virtual space, and a second image that serves as an entrance for moving to various virtual spaces and that depicts an image for a second user who views content in the virtual space; the first image is arranged in a manner that each layer constituting the space forms a vertical plane at a different distance d along a line of sight of a user passing through a predetermined reference axis when viewed from above, The distance d of each layer corresponds to the distance between the respective layer and the user, In the second image, the layers constituting the space are arranged in a manner that they form surfaces of cylinders having different radii r around a predetermined reference axis in a top view, An information processing system in which the radius r of each layer corresponds to the distance between the respective layer and the user.

2. the plurality of layers includes a background layer; The information processing system according to claim 1 , wherein the distance d relating to the background layer can be set to infinity or a distance according to a far clip plane.

3. the plurality of layers includes an interface display layer on which a user interface is displayed; The information processing system of claim 1 or 2, wherein the distance d of the interface display layer can be set to a distance corresponding to a near clip plane, can be set within the reach of the user, and / or can be adjusted for each user.

4. One of the vertical planes in the first image is a first interface display layer; one of the surfaces of the cylinder in the second image includes a second interface display layer; The information processing system according to claim 1 , wherein one distance of the vertical plane is equal to one diameter of the surface of the cylinder.

5. the plurality of layers includes a background layer; The information processing system according to claim 1 , wherein the radius r of the background layer can be set to infinity or infinity.

6. the plurality of layers includes an interface display layer on which a user interface is displayed; 6. The information processing system according to claim 1, wherein the diameter r of the interface display layer is set within the reach of a user and / or can be adjusted for each user.

7. a drawing processing unit that draws an image of a virtual space including an interface display layer on which a user interface is displayed; the interface display layer includes a plurality of operation areas; The multiple operation areas are arranged in a multi-layered manner from front to back, and include a first group arranged in front and a second group arranged behind the first group, and in response to user input, the first group and the second group in the operation area located in front of the user's viewpoint are swapped front to back, but the first group and the second group in the operation area not located in front of the user's viewpoint are not swapped front to back.

8. a drawing processing unit drawing an image of a virtual space having a plurality of layers; The image of the virtual space includes a first image for a first user who delivers content to the virtual space, and a second image that serves as an entrance for moving to various virtual spaces and that depicts an image for a second user who views content in the virtual space; the first image is arranged in a manner that each layer constituting the space forms a vertical plane at a different distance d along a line of sight of a user passing through a predetermined reference axis when viewed from above, The distance d of each layer corresponds to the distance between the respective layer and the user, In the second image, the layers constituting the space are arranged in a manner that they form surfaces of cylinders having different radii r around a predetermined reference axis in a top view, An information processing method, wherein the radius r of each layer corresponds to the distance between the respective layer and the user.

9. causing one or more computers to function as a rendering processing unit that renders an image of a virtual space having a plurality of layers; The image of the virtual space includes a first image for a first user who delivers content to the virtual space, and a second image that serves as an entrance for moving to various virtual spaces and that depicts an image for a second user who views content in the virtual space; the first image is arranged in a manner that each layer constituting the space forms a vertical plane at a different distance d along a line of sight of a user passing through a predetermined reference axis when viewed from above, The distance d of each layer corresponds to the distance between the respective layer and the user, In the second image, the layers constituting the space are arranged in a manner that they form surfaces of cylinders having different radii r around a predetermined reference axis in a top view, An information processing program, wherein the radius r of each layer corresponds to the distance between the respective layer and the user.

10. a drawing processing unit drawing an image of a virtual space including an interface display layer on which a user interface is displayed; the interface display layer includes a plurality of operation areas; The information processing method includes: a plurality of operation areas arranged in a front-to-back multi-layer configuration, each of which includes a first group arranged in front and a second group arranged behind the first group; and a user input causes the first group and the second group in an operation area located in front of the user from the user's viewpoint to be swapped front to back, but does not cause the first group and the second group in an operation area not located in front of the user from the user's viewpoint to be swapped front to back.

11. causing one or more computers to function as a rendering processing unit that renders an image of a virtual space including an interface display layer on which a user interface is displayed; the interface display layer includes a plurality of operation areas; The multiple operation areas are arranged in a multi-layered manner from front to back, and include a first group arranged in front and a second group arranged behind the first group, and in response to a user's input, the first group and the second group are swapped front to back in the operation areas that are located in front of the user from the user's viewpoint in the multiple operation areas, but the first group and the second group are not swapped front to back in the operation areas that are not located in front of the user from the user's viewpoint in the multiple operation areas.

Images