Virtual reality display program and virtual reality provision system

Abstract

To provide a virtual reality display program and a virtual reality provision system, upon executing the dress up of an avatar, by which one's own avatar appearance can be visually recognized without damaging virtual reality immersion.SOLUTION: When an avatar material used for a user-controlled avatar arranged at a virtual reality space is changed, a virtual reality display program causes a computer (10) to realize a function that makes a mirror (902) appear in the virtual reality space and a function that displays an avatar image reflected in the mirror.SELECTED DRAWING: Figure 27

Description

[Technical field]

[0001] The present invention relates to a virtual reality display program and a virtual reality providing system. [Background technology]

[0002] Patent document 1 discloses that "the present invention relates to a method of providing data representative of a body part of a person, the method comprising at least the steps of: providing a head mounted display unit; providing camera means for capturing visual information in front of the person; providing a mirror for reflecting the person wearing the head mounted display unit; capturing the visual information reflected by the camera means; processing the reflected visual information using a processing unit; and outputting the processed visual information to the person (abstract excerpt)." [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Special Publication No. 2020-520487 Summary of the Invention [Problem to be solved by the invention]

[0004] When a user places his / her avatar in a virtual reality space and views a virtual reality image from a so-called first-person viewpoint, where the user's viewpoint is aligned with the avatar's, the user cannot see the avatar's appearance directly. Therefore, when the user changes clothes for the avatar while viewing the virtual reality, the user is unable to see his / her appearance after changing clothes. According to Patent Document 1, the user can see his / her own appearance wearing the head-mounted display unit reflected in a mirror, but the user cannot see the appearance of his / her avatar, and the above problem cannot be solved.

[0005] The present invention has been made in consideration of the above-mentioned circumstances, and its object is to provide a virtual reality display program and a virtual reality provision system that enable a user to visually recognize the appearance of their own avatar when changing the avatar's clothes without compromising the immersive feeling of virtual reality. [Means for solving the problem]

[0006] In order to solve the above problems, the present invention has the configuration described in the claims. As an example, the present invention is a virtual reality display program that causes a computer to realize a function of making a mirror appear in a virtual reality space and a function of displaying an avatar image in which the avatar is reflected in the mirror when an avatar material used for an avatar operated by a user placed in the virtual reality space is changed.

[0007] The present invention also provides a virtual reality provision system configured by communicating between a virtual reality provision server and a user terminal, wherein the user terminal is equipped with an avatar synthesis unit that generates a composite avatar by synthesizing a character model and a clothing model used for an avatar operated by a user placed in a virtual reality space, and the virtual reality provision server is equipped with a user management unit that, when the avatar synthesis unit changes the clothing model used for the avatar, makes a mirror appear near the location of the avatar in the virtual reality space and transmits an avatar image showing the avatar in the mirror to the user terminal. Effect of the Invention

[0008] According to the present invention, it is possible to provide a virtual reality display program and a virtual reality providing system that allow a user to visually recognize the appearance of his / her own avatar without losing the immersive feeling of virtual reality when changing the clothes of the avatar. Objects, configurations, and effects other than those described above will be made clear in the following embodiments. [Brief description of the drawings]

[0009] [Figure 1] A diagram showing an overview of the virtual reality provision system in the first embodiment. [Diagram 2] FIG. 1 is a hardware configuration diagram of a VR provision server. [Diagram 3] FIG. 1 is an external view of an HMD set. [Figure 4] FIG. 2 is a diagram illustrating the hardware configuration of an HMD. [Diagram 5] FIG. 2 is a hardware configuration diagram of a remote controller. [Figure 6] FIG. 1 is a diagram showing the hardware configuration of a PC. [Figure 7] FIG. 2 is a functional block diagram of a VR provision server and a VR user terminal according to the first embodiment. [Figure 8] 11A to 11D are diagrams showing examples of data stored in the VR provision server, where (a) shows a developer management table, (b) shows a character model management table, (c) shows a clothing model management table, and (d) shows a VR user management table. [Figure 9] FIG. 4 is a diagram showing appearance data of a character model. [Figure 10] FIG. 2 is a diagram showing body data of a character model. [Figure 11] FIG. 13 is a diagram showing appearance data of a clothing model. [Figure 12] 11 is an explanatory diagram of a process of making a clothing model follow a character model (avatar synthesis process). FIG. [Figure 13] FIG. 13 is a diagram showing a state in which each character model is combined with a clothing model that is suited to the character model. [Figure 14] This is a sequence diagram showing the flow of the model sales and purchase processing in the virtual reality provision system. [Figure 15] FIG. 13 is a diagram showing an example of a character model sales page. [Figure 16] FIG. 13 is a diagram showing an example of a clothing model sales page. [Figure 17] This is a sequence diagram showing the flow of VR image display processing in the virtual reality provision system. [Figure 18] This is a sequence diagram showing the flow of VR image display processing in the virtual reality provision system. [Figure 19]FIG. 11 is a diagram showing a method for determining coordinates for body data of a character model. [Figure 20] FIG. 13 is a diagram showing an avatar material sales page according to a modification of the first embodiment. [Figure 21] 13A to 13C are diagrams showing examples of data stored in the VR provision server, where (a) shows a developer management table, (b) shows a character model management table, and (c) shows a clothing model management table. [Figure 22] A sequence diagram showing the process of purchasing avatar material for another person's composite avatar in the virtual reality provision system in the second embodiment. [Diagram 23] FIG. 11 is a diagram showing an example of a screen display according to the second embodiment. [Figure 24] FIG. 11 is a diagram showing an example of a screen display according to the second embodiment. [Diagram 25] FIG. 11 is a diagram showing an example of a screen display according to the second embodiment. [Figure 26] FIG. 11 is a diagram showing an example of a screen display according to the second embodiment. [Figure 27] FIG. 13 is a diagram showing an example of a screen display in which a full-body mirror appears. [Figure 28] FIG. 13 is a diagram showing an example of a profile editing screen. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0010] Hereinafter, an embodiment of the present invention will be described with reference to the drawings. Note that the embodiment of the present invention described below is an example of how the present invention is embodied, and the scope of the present invention is not limited to the scope of the embodiment. Therefore, the present invention can be implemented by adding various modifications to the embodiment.

[0011] [Terminology explanation] In the following embodiment, a character model that a user places as a virtual self in a virtual reality space is called an avatar. Three-dimensional shape data (3DCG data) is used as character data for depicting the character model. The character data includes character appearance data indicating the facial expression and three-dimensional shape of the external appearance of the body of the character model, and body data indicating the three-dimensional shape of the physical features of the character model. The body data expresses the positions of body feature points, for example. The body feature points may be, for example, a head center point, a joint point, a clavicle center point, a pelvis center point, a middle point of the upper arm, a middle point of the lower leg, etc. When a shoulder joint point and an elbow joint point are defined as the body feature points, the distance from the shoulder joint point to the elbow joint point corresponds to the length of the upper arm. The setting locations of the body feature points and the data structure of the body data expressing the body data as the body feature points are merely examples, and the data structure of the character model is not limited to the above.

[0012] In addition, three-dimensional shape data (3DCG data) is also used for the clothing model to depict the clothing model to be worn by the character model. The clothing model includes clothing appearance data indicating the three-dimensional shape of the appearance including the size, color, design, etc. of the clothing model, and skeleton tracking data including tracking points indicating the parts of the clothing model that overlap with the body feature points of the character model when the clothing model is worn by the character model.

[0013] First Embodiment [Overview of Virtual Reality Providing System 1] Fig. 1 is a diagram showing an overview of a virtual reality providing system 1 according to this embodiment. As shown in Fig. 1, in the virtual reality providing system 1, at least one, and preferably an unspecified number of character model developer terminals 41a, 41b, ..., 41i, and at least one, and preferably an unspecified number of clothing model developer terminals 42a, 42b, ..., 42j are connected to a virtual reality (hereinafter abbreviated as "VR") providing server 10 via a communication network 4.

[0014] Furthermore, the VR provision server 10 is connected via a communication network 5 to VR user terminals 20a, 20b, and 20c used by users (hereinafter referred to as "VR users") who wish to experience a virtual reality world.

[0015] The numbers of character model developer terminals 41a, 41b, ..., 41i, clothing model developer terminals 42a, 42b, ..., 42j, and VR user terminals 20a, 20b, 20c are not limited to those shown in Fig. 1. Hereinafter, when there is no need to distinguish between the character model developer terminals 41a, 41b, ..., 41i, they may be referred to as "character model developer terminal 41", when there is no need to distinguish between the clothing model developer terminals 42a, 42b, ..., 42j, they may be referred to as "clothing model developer terminal 42", and when there is no need to distinguish between the VR user terminals 20a, 20b, 20c, they may be referred to as "VR user terminal 20".

[0016] Although the communication networks 4 and 5 are illustrated separately for convenience in order to show the connection relationship between the VR provision server 10 and the character model developer terminal 41 and the clothing model developer terminal 42, and between the VR provision server 10 and each VR user terminal 20, the communication networks 4 and 5 may be common. Specific examples of the communication networks 4 and 5 are not particularly limited, and include, for example, the Internet, a mobile communication system (e.g., 4G, 5G, etc.), a wireless network such as Wi-Fi (registered trademark), or a combination of these.

[0017] The VR provision server 10 receives and stores a character model made of 3DCG developed by an external character model developer from the character model developer terminal 41. Similarly, the VR provision server 10 receives and stores a clothing model made of 3DCG that matches a specific character model from an external clothing model developer.

[0018] A clothing model developer obtains the physical data of a specific character model and develops a clothing model that matches it; the method of obtaining this "physical data of a specific character model" may be, for example, from the character model developer via a portable storage medium, or from the character model developer terminal 41 to the clothing model developer terminal 42 via a cloud drive, or as an email attachment; the manner in which this is obtained is not limited, and it does not have to be done within the virtual reality providing system 1.

[0019] Furthermore, the character model and clothing model do not necessarily need to be provided to the VR provision server 10 via the communication network 4, and may be stored in a portable storage medium and imported into the VR provision server 10. Therefore, it is not essential that the character model developer terminal 41 and the clothing model developer terminal 42 are communicatively connected to the VR provision server 10.

[0020] The VR provision server 10 then opens a sales site for selling the usage rights of the stored character models and clothing models, and performs settlement in response to a request to purchase those usage rights from the VR user terminal 20. Thus, the VR provision server 10 has a function as an avatar material sales server.

[0021] The VR provision server 10 displays a specific character model and a clothing model that matches the character model in association with each other on the sales site of the avatar material. This allows the sale of avatar materials while preventing the VR user from mistakenly purchasing a clothing model that does not match the desired character model.

[0022] A VR user uses a composite avatar, which is a character model purchased from an avatar material sales site provided by the VR providing server 10 and dressed in a clothing model, as his / her own avatar, and experiences virtual reality by placing the composite avatar in a virtual reality space provided by the VR providing server 10.

[0023] [VR provision server 10 configuration] 2 is a hardware configuration diagram of the VR provision server 10. The VR provision server 10 is realized by, for example, a general-purpose computer such as a workstation or a personal computer. As shown in FIG. 2, the VR provision server 10 mainly includes a processor 11, a RAM 12, a storage 13, an input / output interface 14, and a communication interface 15. Each component of the VR provision server 10 is connected to a communication bus 19.

[0024] The processor 11 realizes the processes described below by executing a series of instructions included in a server program 13P stored in the RAM 12 or the storage 13. The server program 13P includes an operation system and a virtual reality providing program. The processor 11 is realized as, for example, a central processing unit (CPU), a graphics processing unit (GPU), a field-programmable gate array (FPGA), or other devices.

[0025] The RAM 12 is realized using a RAM (Random Access Memory) or other volatile memory, and is used as a work area during execution of the server program 13P and as a primary storage area for data.

[0026] The storage 13 is realized by using a non-volatile storage device, for example, a removable storage device such as a ROM (Read-Only Memory), a hard disk device, a flash memory, or a memory card. An external non-volatile storage device may also be used as the storage 13. The storage 13 permanently holds the server program 13P and data.

[0027] The input / output interface 14 is an interface for connecting a display, an input device (e.g., a keyboard, a pointing device) 17, a display 18, and other external devices such as an external storage device to the VR provision server 10. The input / output interface 14 is realized using, for example, a Universal Serial Bus (USB), a Digital Visual Interface (DVI), a High-Definition Multimedia Interface (HDMI, registered trademark), or other terminals.

[0028] The communication interface 15 communicates with other devices connected to the communication networks 4 and 5, such as the character model developer terminal 41, the clothing model developer terminal 42, and the VR user terminal 20. The communication interface 15 is realized as, for example, a wired communication interface such as a LAN (Local Area Network) or a wireless communication interface such as Wi-Fi (Wireless Fidelity).

[0029] Although not shown in the figure, the character model developer terminal 41 and the clothing model developer terminal 42, like the VR provision server 10, are composed of computers that mainly include a processor, memory, storage, an input / output interface, and a communication interface, all of which are connected to each other by a communication bus.

[0030] [Configuration of VR user terminal 20] In this embodiment, an HMD set 200 is used as the VR user terminal 20. Fig. 3 is an external view of the HMD set 220. As shown in Fig. 3, the HMD set 200 includes an HMD 210, a remote controller 230, and a personal computer (hereinafter abbreviated as "PC") 30, and is configured by connecting these for communication.

[0031] The HMD 210 includes an HMD body 211 and a holder 219 for holding the HMD body 211 in front of the user's eyes. The HMD body 211 includes an HMD-side short-range wireless communication device 213 (see FIG. 4). As an example of the HMD-side short-range wireless communication device 213 (see FIG. 4), a Bluetooth (registered trademark, hereinafter abbreviated as "BT") wireless device may be used.

[0032] The remote controller 230 includes a remote control side short-range wireless communication device 221 in a housing 229, and a rear button 222 on the rear surface of the housing 229. The remote control side short-range wireless communication device 221 is, for example, a BT communication device (see FIG. 5), and is communicatively connected to the HMD side short-range wireless communication device 213 or the PC 30. When the HMD 210 performs the image generation process, the HMD 210 may directly receive an operation signal from the remote controller 230 and reflect it in the image displayed on the HMD 210. Also, if the PC 30 performs the image generation process and the HMD 210 receives and displays the image signal, the operation signal from the remote controller 230 may be configured to be received by the PC 30. In this embodiment, the latter is taken as an example and the following description will be given.

[0033] The PC 30 is configured using, for example, a desktop personal computer. The PC 30 is communicatively connected to the VR provision server 10 via a communication network 5 such as the Internet. The PC 30 receives virtual reality rendering data from the VR provision server 10, generates virtual reality video data, and transmits the data to the HMD 210 from the PC 30.

[0034] [HMD210 Configuration] 4 is a hardware configuration diagram of the HMD 210. The HMD 210 includes a processor 101, a RAM 102, a display 103, a storage 104, an input interface 105, a motion sensor 106, an outer camera 107, a communication interface 108, a microphone 109, and a speaker 110 in an HMD main body 211, which are connected to each other via a communication bus 111.

[0035] The processor 101 realizes the processes described below by executing a series of instructions included in an HMD control program 104P stored in the storage 104. The processor 101 is realized as, for example, a central processing unit (CPU), a graphics processing unit (GPU), a microprocessor unit (MPU), a field-programmable gate array (FPGA), or other devices.

[0036] The RAM (Random Access Memory) 102 is a volatile memory.

[0037] As an example, the display 103 is realized as a non-transmissive display device. The display 103 is disposed on the HMD body 211 so as to be located in front of both eyes of the user. The non-transmissive display 103 is realized as, for example, a liquid crystal monitor or an organic EL (Electro Luminescence) monitor.

[0038] As another example, the display 103 is realized as a transmissive display device. In this case, the HMD 210 is not a closed type that covers the user's eyes, but an open type such as a glasses type. The display 103 may include a configuration that simultaneously displays a part of an image that constitutes a virtual reality space and the real space. As an example, the transmissive display 103 may display an image of the real space captured by a camera mounted on the HMD 210. As another example, the transmissive display 103 may be configured to have an adjustable transmittance. The transmissive display 103 may set the transmittance of a part of the display area high so that the real space can be directly viewed.

[0039] Furthermore, the display 103 may employ the following configuration to allow the user to view a three-dimensional image. As an example, the display 103 may include a sub-monitor for displaying an image for the right eye and a sub-monitor for displaying an image for the left eye. As another example, the display 103 may be configured to integrally display an image for the right eye and an image for the left eye. In this case, the display 103 includes a high-speed shutter. The high-speed shutter operates to alternately display an image for the right eye and an image for the left eye so that the image is recognized by only one of the eyes.

[0040] The storage 104 is a storage medium that permanently holds data, and is realized as, for example, a ROM (Read-Only Memory), a flash memory, or other non-volatile storage medium. The storage 104 may also be realized as a removable storage medium such as a memory card. The storage 104 stores an HMD control program 104P required for controlling the HMD 210.

[0041] The input interface 105 is, for example, a USB (Universal Serial Bus) terminal.

[0042] The motion sensor 106 detects the movement of the HMD 210, for example, the movement of the user's head when the HMD 210 is worn on the user's head, because the HMD 210 and the user's head move almost together. The motion sensor 106 is realized by using, for example, a three-axis acceleration sensor.

[0043] The outer camera 107 is a camera that captures an external world image. In the immersive HMD 210, a through image of the outer camera 107 is displayed on the display 103, so that the user can visually recognize the external world image while wearing the HMD 210 and understand the situation around him / her.

[0044] The communication interface 108 is an interface for communicatively connecting the HMD 210 and an external device. As an example, the communication interface 108 may be configured as a USB terminal, and a BT dongle may be used as the HMD-side short-range wireless communication device 213 to perform short-range wireless communication with the remote controller 230 or the PC 30. Simultaneous connection between the remote controller 230 and the PC 30 may be realized by using a BT communication device compatible with multi-pairing.

[0045] The microphone 109 is a device that collects sound around the HMD 210. When the user inputs voice, the microphone 109 collects the user's voice and inputs it to the HMD 210, where voice recognition processing is executed, thereby enabling voice input.

[0046] The speaker 110 is an audio output device that outputs the audio of the content being displayed on the display 103 and other audio.

[0047] [Configuration of the remote controller 230] 5 is a hardware configuration diagram of the remote controller 230. The remote controller 230 includes a processor 201, a RAM 202, a ROM 203, an operation switch 204, a motion sensor 205, and a BT communication device 206, which are connected to each other via a communication bus 209.

[0048] The RAM 202 is a volatile memory and is used as a temporary storage area for processing programs and data.

[0049] The ROM 203 stores a processing program for the remote controller 230 .

[0050] The operation switch 204 includes, for example, a rear button 222, a stick switch (not shown), a track pad, and the like.

[0051] The motion sensor 205 is a sensor that detects the movement of the remote controller 230, and is realized using, for example, a three-axis acceleration sensor.

[0052] The processor 201 is configured using, for example, an MPU (Micro Processor Unit), and reads out a control program for the remote controller 230 from the ROM 203, loads it into the RAM 202, and executes it. The processor 201 transmits operation information of the operation switch 204, the movement of the remote controller 230 detected by the motion sensor 205, and the like to the PC 30 via the BT communication device 206. Therefore, the BT communication device 206 corresponds to the remote control side short-range wireless communication device 221.

[0053] [PC30 Configuration] 6 is a diagram showing the hardware configuration of the PC 30. The PC 30 includes a processor 301, a RAM 302, a ROM 303, a storage 304, an input interface 305, and a communication interface 306, which are connected to each other via a communication bus 309. Details of each part overlap with those of the HMD 210 and the remote controller 230, and will not be described here.

[0054] The processor 301 is realized, for example, by using a GPU (Graphics Processing Unit).

[0055] The RAM 302 is a volatile memory and is used as a temporary storage area for processing programs and data.

[0056] The ROM 303 is a non-volatile memory that stores the operating system program of the PC and the like.

[0057] The storage 304 is realized by a non-volatile storage medium such as a hard disk drive (HDD) or a solid state drive (SSD), and stores the virtual reality display program and the avatar synthesis program according to this embodiment.

[0058] [Functional block diagram of the VR provision server 10 and the VR user terminal 20] Fig. 7 is a functional block diagram of the VR provision server 10 and the VR user terminal 20 (PC 30). As shown in Fig. 7, the server program 13P loaded into the RAM 12 of the VR provision server 10 causes the VR provision server 10 to function as a login authentication unit 401, a character model management unit 402, a clothing model management unit 403, a user management unit 404, a sales page control unit 405, a VR drawing data transmission unit 406, a purchase reception unit 407, and a settlement unit 408. In addition, the storage 104 includes a character model storage unit 409, a clothing model storage unit 410, a VR user data storage unit 411, and a developer management data storage unit 412.

[0059] Furthermore, the terminal program 304P loaded into the RAM 302 of the PC 30 of the VR user terminal 20 causes the PC 30 to function as a VR video generation unit 431, an avatar synthesis unit 432, and an avatar action control unit 433. The functions of these units will be described later.

[0060] Figure 8 is a diagram showing an example of data stored in the VR provision server 10, where (a) is a developer management table, (b) is a character model management table, (c) is a clothing model management table, and (d) is a diagram showing an example of data stored in the VR provision server 10, showing a VR user management table.

[0061] Developer management table 500 shown in FIG. 8(a) is stored in developer management data storage unit 412. Developer management table 500 stores, in association with each developer, a "developer ID" that uniquely identifies a character model developer and a clothing model developer, a "PIN code" that the developer uses when logging in, the developer's name "NAME," and an "attribute" that identifies whether the developer is a character model developer (flag 1) or a clothing model developer (flag 2). Furthermore, a configuration may be used in which a developer receives a predetermined percentage of sales when sales of the usage rights for a character model or clothing model that the developer developed are made. In that case, developer management table 500 may further include "account information" that includes account information that each developer uses for payment.

[0062] Character model management table 520 shown in Fig. 8(b) is stored in character model storage unit 409. Character model management table 520 stores a "Model ID" (corresponding to character model identification information) that uniquely identifies a character model, a "developer ID" that indicates the developer of that character model, and "CharaModel DATA," the model data of that character model, in association with each other. In reality, the address of the storage destination of "CharaModel DATA" is written in character model management table 520, and the actual data of "CharaModel DATA" is not included in character model management table 520, but it can be said that it is indirectly associated with and stored in character model management table 520.

[0063] "CharaModel DATA" is data in which physical data is superimposed on the appearance data (shape, color, etc.) of a character model. The physical data is data in which the physical feature points of the character model are defined as a point cloud.

[0064] 8(c) is stored in the clothing model storage unit 410. The clothing model management table 540 stores an "Outfit ID" (corresponding to clothing model identification information) that uniquely identifies a clothing model, a "Developer ID" that indicates the developer of the clothing model, "OutfitModel DATA" that is the model data of the clothing model, "Size" that indicates the size of the clothing model, and a CharaModel ID of a "compatible model" that indicates a character model that the clothing model is compatible with, in association with each Outfit ID. As with the character model, the clothing model management table 540 actually stores the address of the storage destination of "OutfitModel DATA", and although the actual data of "OutfitModel DATA" is not included in the clothing model management table 540, it can be said that it is indirectly associated with and stored in the clothing model management table 540.

[0065] "OutfitModelDATA" is data in which the appearance data (shape, color, etc.) of a clothing model is superimposed with point cloud data including tracking points that track the body data of the character model to which the clothing model is compatible, and feature points of the clothing model.

[0066] The VR user management table 560 shown in FIG. 8(d) is stored in the VR user data storage unit 411. The VR user management table 560 stores, in association with the VR user, a "VR user ID" that uniquely identifies the VR user, a "PIN code" that the VR user uses when logging in, the name of the VR user "NAME", an "Avatar Model" that indicates the character model that the VR user is currently using for his / her avatar, an "Avatar Outfit" that indicates the clothing model that the VR user is to wear for the character model that the VR user is currently using for his / her avatar, "Account Information" that includes account information used for settlement when each VR user purchases the usage rights for the character model and clothing model, and "Owned Materials" that indicates the model for which the usage rights for data have been purchased. In the "Avatar Model", a "Model ID" that uniquely indicates the character selected by the VR user is input, and the character model can be referenced by referring to the "Model ID" in the character model management table 520. Similarly, an “Outfit ID” that uniquely identifies the clothing model selected by the VR user is entered into “Avatar Outfit”, and the clothing model can be referenced by referring to the “Outfit ID” in the clothing model management table 540.

[0067] FIG. 9 is a diagram showing appearance data of a character model, and FIG. 10 is a diagram showing body data of the character model.

[0068] As shown in Fig. 9, the appearance data of a character model is composed of data expressed in 3DCG that shows the facial expression and physical appearance of the character. Fig. 9 shows four character models, "Model 1" to "Model 4", and the height and build of the character models increase in order from "Model 1" to "Model 4". "Model 1" is suitable for a clothing size of S, "Model 2" is suitable for a clothing size of M, and "Model 4" is suitable for a clothing size of L.

[0069] As shown in FIG. 10, the body data of the character model is composed of 3DCG data in which the body feature points of the character model, for example the head and each joint point, are defined as a point cloud.

[0070] FIG. 11 is a diagram showing the appearance data of a clothing model, FIG. 12 is an explanatory diagram of the process of matching a clothing model to a character model (avatar synthesis process), and FIG. 13 is a diagram showing the state in which each character model is synthesized with a clothing model that is compatible with that character model.

[0071] FIG. 11 shows appearance data "Outfit 1" of a model wearing size S clothing, appearance data "Outfit 2" and "Outfit 3" of a model wearing size M clothing, and appearance data "Outfit 4" of a model wearing size L clothing.

[0072] 12, the clothing model "Outfit 1" includes tracking points "cn" (n=1,2,...,12) that indicate parts of the clothing model that overlap with the body feature points "Jn" (n=1,2,...,12) of the character model "Model 1" when the clothing model "Outfit 1" is put on the body data of the character model "Model 1." In other words, "cn" (n=1,2,...,12) are defined at positions that correspond to the "Jn" (n=1,2,...,12) provided in the body data of Model 1, respectively.

[0073] Therefore, by superimposing each point of “cn” (n = 1, 2, . . . , 12) on each point of “Jn” (n = 1, 2, . . . , 12) provided in the body data of Model 1, an avatar can be synthesized in which the appearance of each character model matches the appearance of the clothing model, as shown in Figure 13.

[0074] As shown in Figures 9, 10, and 11, clothing size categories are defined by setting overlapping ranges for the height of the character model between S and M, and between M and L. This means that, for example, "Outfit 2" of size M created to fit "Model 2" also fits "Model 3," and "Outfit 3" of size M created to fit "Model 3" also fits "Model 2."

[0075] 14 is a sequence diagram showing the flow of the process of selling and purchasing a model in the virtual reality provision system 1. It is a sequence diagram showing the flow of the process of the virtual reality provision system 1 in an embodiment. As a prerequisite for starting the process of FIG. 14, it is assumed that each developer has registered in the VR provision server 10, and that the registration information has been registered in the developer management table 500.

[0076] The character model developer operates the character model developer terminal 41 to send login information to the VR provision server 10 (S01), and the login authentication unit 401 of the VR provision server 10 compares the login information with the registration information in the developer management table 500 to perform login authentication processing (S02).

[0077] The character model developer operates the character model developer terminal 41 to upload the appearance data and body data of the character model to the VR provision server 10 (S03), and in the VR provision server 10, the character model management unit 402 stores the uploaded character data in the character model storage unit 409 and updates the character model management table 520 (S04).

[0078] The clothing model developer operates the clothing model developer terminal 42 to send login information to the VR provision server 10 (S05), and the login authentication unit 401 of the VR provision server 10 compares the login information with the registration information in the developer management table 500 to perform login authentication processing (S06).

[0079] The clothing model developer operates the clothing model developer terminal 42 to upload the completed clothing model data to the VR provision server 10 (S07). At that time, information indicating which character model the clothing model matches is also added and uploaded. In the VR provision server 10, the clothing model management unit 403 stores the uploaded clothing model data in the clothing model storage unit 410, and updates the clothing model management table 540 (S08). Updating the clothing model management table 540 also records information indicating which character data the newly stored clothing model data matches.

[0080] When new character data or clothing model is stored, the sales page control unit 405 of the VR provision server 10 generates an avatar material sales page to which the newly stored character data or clothing model is added, and publishes it (S09). Therefore, step S09 corresponds to a sales page generation step. The clothing model sales page lists explanatory information of the target character model that is the subject of sale on the page, and explanatory information of a clothing model that suits the character, and does not list explanatory information of a clothing model that does not suit the character.

[0081] For example, the sales page control unit 405 refers to the "Model ID" in the character model management table 520 to select a target character model. Then, it extracts clothing models whose "Model ID" of the target character model is stored in the "matching model" in the clothing model management table 540. Then, only the extracted clothing models are posted on the sales page of the target character model. In this way, the sales page control unit 405 generates a sales page in which clothing models that are matched to the target character model are posted, and clothing models that are not matched are not posted.

[0082] FIG. 15 is a diagram showing an example of a character model sales page, and FIG. 16 is a diagram showing an example of a clothing model sales page.

[0083] The sales page control unit 405 generates a character model sales page that displays character models that can be sold by the VR provision server 10. The VR user selects a character model for which the data usage right of the character model is to be purchased from the character model sales page (FIG. 15) and transmits character model selection information. The selection operation may be performed by aligning the pointer 240 of the remote controller 230 with the character model and performing a confirmation operation, for example, by pressing the back button 222.

[0084] Next, when the sales page control unit 405 receives the character model selection information, it generates and displays a clothing model sales page (FIG. 16) on which the selected character model and a clothing model to be matched with it are extracted and listed side by side.

[0085] The character model sales page and the clothing sales page may be generated and displayed as an inventory screen displayed in the virtual reality space, or may be generated and displayed as a Web page. As another example of the avatar material sales page, a page similar to the avatar material sales page (see FIG. 20) according to a modified example of the first embodiment described later may be created. However, while the URL information of the external server is described in the explanatory information of the clothing model in FIG. 20, it is not necessary to describe the URL information of the external server on the sales page of the avatar material stored in the VR provision server 10.

[0086] The VR user refers to the avatar material sales page, and if he / she wishes to make a purchase, he / she logs in to the VR provision server 10 (S10). If the login authentication is successful (S11), he / she performs an operation to purchase the usage right of the character model or a clothing model that matches the character model on the character model sales page (FIG. 15) or clothing model sales page (FIG. 16), and transmits payment information (S12).

[0087] The settlement unit 408 of the VR provision server 10 executes settlement processing, and the user management unit 404 writes information on the purchased character model or clothing model (for example, the ID of the purchased model) to "owned materials" in the VR user management table 560 (S13). This allows the VR user to use the purchased character model or clothing model as his or her own avatar in the future.

[0088] The process flow of the VR user terminal and the VR provision server in the virtual reality provision system 1 will be described with reference to Fig. 17 and Fig. 18. Fig. 17 and Fig. 18 are sequence diagrams showing the process flow of VR video display in the virtual reality provision system.

[0089] As shown in FIG. 17, the VR user operates the VR user terminal 20 to send login information to the VR providing server 10 (S21), and the login authentication unit 401 of the VR providing server 10 compares the VR user registration information with the login information to perform login authentication processing (S22).

[0090] The VR drawing data transmission unit 406 of the VR provision server 10 transmits drawing data necessary for the generation process of the VR video to the VR user terminal 20 (S23), and the VR user terminal 20 generates the VR video based on the drawing data, and the VR video is displayed on the display 214 (S24). The VR video may be a virtual reality video (VR video) from the viewpoint of the avatar (first-person viewpoint) or a viewpoint different from the viewpoint of the avatar, for example, a viewpoint watching from behind (third-person viewpoint). The drawing data necessary for the generation process of the VR video also includes drawing data necessary for drawing other people's avatars.

[0091] Next, the VR user selects an avatar as the virtual self to be placed in the VR space. The VR user operates the VR user terminal 20 to transmit character model inquiry information to the VR provision server 10 (S25). The user management unit 404 of the VR provision server 10 refers to the "owned materials" in the VR user management table 560, and generates and displays a character model selection screen (not shown) in which images of purchased character models are arranged (S26). As an example of the layout of the character model selection screen, for example, a layout similar to that of the character model sales page in FIG. 15 may be used. However, by limiting the display to the character models included in the "owned materials", only the purchased character models are displayed. The same applies to clothing models, and a layout similar to that of the clothing model sales page in FIG. 16 may be used.

[0092] The VR user selects a character model that he or she wishes to use as his or her avatar from the character model selection screen by operating the remote controller 230 of the VR user terminal 20, and transmits character model selection information (S27).

[0093] The user management unit 404 reads out the selected character model, registers the Model ID of the selected character model in "Avatar Model" of the VR user management table 560 (S28), and transmits the character model to the VR user terminal 20 (S29).

[0094] Next, the VR user operates the VR user terminal 20 to send clothing model inquiry information to the VR provision server 10 (S30). The user management unit 404 refers to the "owned materials" in the VR user management table 560, and extracts clothing models that match the character model selected by the VR user from the clothing model storage unit and displays them (S31). The clothing model management unit 403 generates and displays a clothing model selection screen (FIG. 17) that lists the clothing appearance data of the extracted clothing models (S31).

[0095] The VR user operates the remote controller 230 of the VR user terminal 20 to select a clothing model that he / she wants to dress his / her avatar character model from the clothing model selection screen, and transmits clothing model selection information (S32).

[0096] The user management unit 404 registers the outfit ID of the selected clothing model in the "Avatar Outfit" field of the VR user management table 560 (S33), and transmits the clothing model to the VR user terminal 20 (S34).

[0097] The avatar synthesis unit of the VR user terminal 20 associates the body feature points of the character model with the corresponding tracking points of the clothing model, synthesizes a composite avatar by dressing the character model in the clothing model (S35), and displays it on the HMD 210 (S36).

[0098] FIG. 18 is a diagram showing a method for determining coordinates for the body data of a character model.

[0099] In FIG. 18, the character "Model 1" is taken as an example for explanation. In the body data "CharaModel DATA" of the character model "Model 1", M10 (x0, y0, z0) is defined as the reference point of the character model at the center of the head of the character model "Model 1". The coordinates of each joint point J1, J2, J3, J4, J5, J6, J7, J8, J9, J10, J11, and J12 provided on the body data are defined in relative coordinates based on M10 (x0, y0, z0). In this case, the character "Model 1" is made to take a basic posture in which it stands upright, and the displacements of each joint point J1, J2, J3, J4, J5, J6, J7, J8, J9, J10, J11, and J12 as viewed from the reference point M10 (x0, y0, z0) in the three-axis character model coordinate system (xyz coordinate system) are used to define the position. J1(x J1 ,y J1 ,z J1 )=(x0+Δx J1 ,y0+Δy J1 ,z0+Δz J1 ) J2(x J2 ,y J2 ,z J2 )=(x0+Δx J2 ,y0+Δy J2 ,z0+Δz J2 ) ... J12(x J12 ,y J12 ,z J12 )=(x0+Δx J12 ,y0+Δy J12 ,z0+Δz J12 )

[0100] The clothing model "Outfit 1S" has been developed to fit the body data of "Model 1." As mentioned above, the clothing model has defined tracking points C1, C2, C3, C4, C5, C6, C7, SC, C10, C11, and C12 that correspond to joint points J1, J2, J3, J4, J5, J6, J7, J8, J9, J10, J11, and J12, which are body feature points on the body data of the character model "Model 1."

[0101] The avatar synthesis unit associates the coordinates of points C1, C2, . . . , C12 defined in the "OutfitModelDATA" of "Outfit 1S" with the joint points J1, J2, . . . , J12 of the character model "Model 1". That is, C1(x C1 ,y C1 ,z C1 )=J1(x J1 ,y J1 ,z J1 )=(x0+Δx J1 ,y0+Δy J1 ,z0+Δz J1 ), C2(x C2 ,y C2 ,z C2 )=J2(x J2 ,y J2 ,z J2 )=(x0+Δx J2 ,y0+Δy J2 ,z0+Δz J2 ), ... C12(xC12 ,y C12 ,z C12 )=J11(x J11 ,y J11 ,z J11 )=(x0+Δx J11 ,y0+Δy J11 ,z0+Δz J11 ) Correspond to:

[0102] When placing the character model "Model 1" in the virtual reality space, the coordinates of the reference point M10 (x0, y0, z0) of the character model are rewritten to three-dimensional coordinates (Xvr1, Yvr1, Zvr1) in the virtual reality space. As a result, the coordinates of each joint point J1, J2, ..., J12 of the character model "Model 1" are rewritten to three-dimensional coordinates in the virtual reality space. Furthermore, the coordinates of each tracking point C1, C2, ..., C12 on the clothing model are also rewritten to the same three-dimensional coordinates in the virtual reality space as the corresponding joint points J1, J2, ..., J12 of the character model. This allows the movement of the clothing model to follow the movement of the character model "Model 1".

[0103] In the VR user terminal 20, the avatar operation control unit 433 controls the operation of the avatar to follow the user's movement based on the sensor information output by the motion sensor that detects the user's movement (S37), and an image of the avatar that moves following the user's movement is displayed in the VR image displayed on the display 103 of the HMD 210 (S38). When the VR user terminal 20 logs out, the process ends.

[0104] [Effects of this embodiment] According to the VR provision server 10 of this embodiment, many character models and clothing models can be acquired from an unspecified number of character model developers and clothing model developers, and sold. In this case, there is a concern that general users may mistakenly purchase a character model and a clothing model that does not match the character model, due to a data structure specific to avatar materials, in which the character model cannot be changed unless the clothing model matches the body data of the character model.

[0105] In this embodiment, on the clothing model sales page, a character model and a clothing model that matches the character model are displayed, and clothing models that do not match the character model are not displayed, so that the user can purchase the clothing model that he or she wants to wear on the character model without making a mistake.

[0106] Furthermore, the avatar synthesis unit of the VR user terminal can automatically associate the body feature points of the character model with the tracking points of the clothing model to create a composite avatar. This allows the VR user to enjoy changing the clothes of the character model without having to manually associate the body feature points of the character model with the tracking points of the clothing model, as was previously the case.

[0107] Furthermore, by creating the character model and the clothing model using a programming language used in a general-purpose game engine and implementing an avatar synthesis unit in the VR user terminal 20 using a game engine using that programming language, the VR user terminal 20 can generate and display a composite avatar that combines the character model and the clothing model without having to install developer software used by character model developers and clothing model developers. Unity, for example, may be used as the programming language used in the general-purpose game engine.

[0108] [Example of change] In the above embodiment, an example was described in which clothing models suitable for a target character model are extracted from clothing models stored in the VR provision server 10, and a clothing model sales page is generated and displayed for purchase. However, the method of generating an avatar material sales page that includes clothing models suitable for a target character model and does not display clothing models that do not match is not limited to the above.

[0109] For example, the sales page control unit 405 accesses an external server different from the VR provision server 10, and collects information on clothing models being sold on an external sales page published on the external server. As a form of collection, the collection may include a description (text information) of the clothing model, image information of the clothing model, and URL information of the external sales page on which the clothing model is posted. This process corresponds to an information collection step.

[0110] The sales page control unit 405 may then classify the collected data by the name of the character model and generate a sales page for each character model. Fig. 20 shows an avatar material sales page according to a modification of the first embodiment.

[0111] On the avatar material sales page 600, an avatar image 601 using a character model, the name 602 of the character model, and explanatory information 611, 612, 613 of clothing models that match the character model are displayed side by side. The explanatory information 611, 612, ... for each clothing model, the price of the clothing model, and a URL for jumping to an external server selling the clothing model are listed.

[0112] According to this modification, even if the avatar material is sold by an external server different from the VR provision server 10, it is possible to purchase a clothing model that matches the character model without making a mistake.

[0113] <Second embodiment> The second embodiment is an embodiment in which, in the virtual reality providing system 1 of the first embodiment, avatar materials used for avatars used by other people in the virtual reality space provided by the VR providing server 10 can be purchased.

[0114] [Character model management table and clothing model management table configuration] FIG. 21 is a diagram showing an example of data stored in the VR provision server, where (a) shows a character model management table 520a, (b) shows a clothing model management table 540a, and (c) shows a VR user management table 560a.

[0115] The character model management unit 402 registers and updates Model L in the character model management table 520 of the first embodiment.

[0116] The clothing model management unit 403 also registers and updates Outfit XS, Outfit XM, and Outfit XL in the clothing model management table 540 of the first embodiment.

[0117] In addition, the user management unit 404 adds user3 to the VR user management table 560 of the first embodiment, and further adds an "Avatar Name" record for each user's avatar display name, and adds and updates an "Avatar Position Information" record indicating the location of each avatar in the virtual reality space.

[0118] A composite avatar is composed of a character model indicated by the Model ID stored in "Avatar Model" and a clothing model indicated by the Outfit ID stored in "Avatar Outfit." In other words, a composite avatar is composed by dressing a character model indicated by the Model ID stored in "Avatar Model" with a clothing model indicated by the Outfit ID stored in "Avatar Outfit."

[0119] The "avatar position information" indicates the coordinates where the composite avatar exists, using the name of the world indicating the virtual reality space where the avatar exists and the coordinates defined in the world coordinate system in the world. The world in the "avatar position information" is a concept for identifying each virtual reality space, and avatars in the same world can meet, but avatars in different worlds cannot meet unless they move to the same world. The coordinates of the avatar position information indicate a point expressed in a three-axis orthogonal coordinate system of each world, and the point is the coordinate of the reference point M10 of each avatar in FIG. 19. In the example of the VR user management table 560a, the drawing data of the virtual reality space identified in world 1 is delivered to the VR user terminal 20a, and the drawing data of the virtual reality space identified in world 2 is delivered to the VR user terminals 20b and 20c. For example, acquaintance VR users may form a party, place only the composite avatars of the party members in one world, and experience virtual reality by synchronizing the virtual reality space image within the party.

[0120] In this embodiment, as long as the virtual reality space (world) is provided by the VR provision server 10, the user can use the composite avatar not only in a virtual reality space that is a copy of the same virtual reality space, but also in different virtual reality spaces. For example, both world 1 and world 2 may be virtual reality spaces that reproduce a festival venue in city space A, or world 1 may be a festival venue in city space A, and world 2 may be a virtual travel space in region B. One of the features of this embodiment is that the composite avatar can be carried over to different virtual reality spaces (worlds) provided by the VR provision server 10.

[0121] FIG. 22 is a sequence diagram showing the flow of a process for purchasing avatar material for another person's composite avatar in the virtual reality providing system 1 in the second embodiment. As a premise for the following explanation, it is assumed that VR users B and C are watching VR video provided by the VR providing server 10 using the VR user terminals 20b and 20c. VR users B and C place avatars B and C in the same VR space (world 2). Avatars B and C are composite avatars that combine a character model according to the first embodiment with a clothing model. The character model and clothing model are components of the composite avatar and correspond to avatar material. In the following, an example will be described in which VR user B purchases avatar material used for avatar C. Avatar C corresponds to another person's avatar from the perspective of VR user B.

[0122] When the VR user B encounters the avatar C and wishes to purchase the avatar material used in the avatar C, the VR user B selects the avatar C, and the selection information is transmitted to the VR provision server 10 (S51).

[0123] The purchase reception unit 407 reads the information on the character model and clothing model written in the "Avatar Model" and "Avatar Outfit" columns for Avatar C in the VR user management table 560a (S52).

[0124] The purchase reception unit 407 generates a details screen (inventory) of Avatar C that displays the materials used in Avatar C based on the original data of the combination (S53), and transmits it to the VR user terminal 20b (S54).

[0125] The VR user terminal B selects the material that the user wishes to purchase on the details screen of avatar C and performs a purchase request operation (S55), the purchase request information is sent to the purchase reception unit 407 (S56), and the payment process for the price of the selected material is executed by the payment unit 408 (S57).

[0126] When the payment process by the payment unit 408 is completed, the user management unit 404 updates the information of the newly purchased avatar material in the VR user management table 560a of VR user B (S58).

[0127] 23 to 26 are diagrams showing examples of screen display according to the second embodiment. As shown in Fig. 23, when an avatar C801 used by another person, in this example, a VR user C, is selected using the remote controller 230 of the VR user terminal B on a screen 800 displaying a VR space, the avatar C801 becomes in a selected state, and the purchase reception unit 407 displays a purchase screen icon 802 for transitioning to a purchase mode. In this example, the text "Buy now" is displayed as the purchase screen icon 802.

[0128] When the remote controller 230 of the VR user terminal B is operated to operate the purchase screen icon 802, a transition to a screen 810 in FIG. 24 is made. On the screen 810, an inventory display icon 811 for displaying the inventory of the avatar C 801 and a menu screen 812 are displayed. The text "Take out / put away inventory" is displayed as the inventory display icon 811. When an inventory display operation is performed using the remote controller 230 of the VR user terminal B, a transition to a screen 820 in FIG. 25 is made.

[0129] The screen 820 displays an inventory screen 821 that displays the material of the avatar C801. The inventory screen 821 displays a clothing model 822 as an example of the avatar material used for the avatar C801. Furthermore, the inventory screen 821 displays a size selection icon 823 for selecting a clothing model of the same design as the clothing model 822 and of various sizes as purchasing information for the clothing model 822. Furthermore, an advice message 824 may be displayed that indicates which clothing model is suitable for the character model currently being used by the VR user terminal B. When a clothing model size selection operation is performed using the remote controller 230 of the VR user terminal B, a transition to a screen 830 in FIG. 26 is made.

[0130] A purchase confirmation icon 831 is displayed on screen 830. When the purchase confirmation icon 831 is operated using the remote controller 230 of VR user terminal B to complete the payment, a purchase completion screen 832 is displayed. When a "Yes" icon 833 is operated using the remote controller 230 of VR user terminal B in response to "Do you want to use it now?", the clothing of the composite avatar currently being used by VR user terminal B is switched to clothing model 822.

[0131] In the above, an example of purchasing a clothing model worn by another person's composite avatar was given, but it is also possible to purchase a character model used in another person's composite avatar. In this case, the character model used in the other person's composite avatar may be displayed on the inventory screen in FIG. 25 and may be purchased.

[0132] According to this embodiment, it is possible to purchase avatar materials, such as character models and clothing models, used in composite avatars of people one meets in virtual reality.

[0133] At this time, if the body model of the character model used in one's composite model is different from that of the character model used in another person's composite model, one can select and purchase the clothing model he or she needs from clothing models of different sizes.

[0134] Furthermore, when multiple sizes are proposed for a clothing model, the user can make a purchasing decision by referring to advice on which clothing model has the size that suits the character model that the user is using.

[0135] <Third embodiment> The third embodiment is an embodiment in which, when an avatar material is purchased in the virtual reality providing system 1 of the first and second embodiments, the appearance of the user's avatar is confirmed and the appearance of the avatar is converted into a profile image of the user's avatar. The profile information of each avatar may be stored in the VR user data storage unit 411 or may be stored in the VR user terminal 20.

[0136] While experiencing virtual reality by placing one's avatar in a virtual reality space, a so-called first-person viewpoint VR image in which the viewpoint of the avatar and one's viewpoint are aligned so that one's movements are reflected in the avatar is displayed on the HMD 210. Therefore, in the first and second embodiments, even if one purchases avatar materials, for example, clothing models and changes clothes, one cannot directly view one's whole body image.

[0137] Therefore, in this embodiment, when the user changes clothes for his / her avatar, a full-length mirror (a so-called full-length mirror) appears. Fig. 27 is a diagram showing an example of a screen display in which a full-length mirror appears.

[0138] In the screen 900 of FIG. 27, a mirror 902 appears in the middle of a passage 901 in a virtual reality space. An example of a trigger for the appearance of the mirror 902 may be when the user management unit 404 updates the VR user management table 560a because the avatar 903 has changed clothes. The avatar 903 changes clothes, for example, when the user operates the "Yes" icon 833 in the screen 830 of FIG. 26 in response to "Do you want to use it now?". When the avatar changes clothes, the user management unit 404 makes the mirror 902 appear in a place in the virtual reality space where the mirror 902 is not originally installed. Note that the user management unit 404 performs the operation only for the VR user terminal 20 that makes the mirror 902 appear (e.g., 20B) operated by the user who changed the clothes of the avatar 903, and when there is another user who has entered the same world, it is not necessary to perform the operation for the VR user terminal (e.g., 20C) used by the other user. This makes it possible to prevent a user who is not dressing up their avatar from being obstructed in the field of vision or losing the sense of immersion due to the appearance of mirror 902 in the virtual reality viewed by the user.

[0139] User management unit 404 displays the avatar image after changing the clothes of the clothing model by fitting it in mirror 902. The user visually checks the appearance of his / her own avatar 903 reflected in mirror 902 and performs a profile reflection operation.

[0140] An example of the profile reflection operation may be an operation of aligning the pointer of the remote controller 230 with the mirror 902 and pressing the back button 222. When the profile reflection operation is performed, a transition is made from the screen 900 in Fig. 27 to the screen 910 in Fig. 28. The appeared mirror 902 may disappear when the profile reflection operation is completed, or a separate operation may be performed to make the mirror 902 disappear. Fig. 28 is a diagram showing an example of a profile editing screen.

[0141] The user management unit 404 displays an avatar profile screen 911. Then, the avatar image 912 in the profile screen 911 is replaced with a full-body image of the mirror 902.

[0142] Avatar image 912 in profile screen 911 may be linked to avatar image 601 on avatar material sales page 600 in FIG. 20, so that when avatar image 912 is replaced, sales page control unit 405 points to avatar image 601 on avatar material sales page 600 instead.

[0143] According to this embodiment, when changing the clothes of an avatar, the user cannot see his / her own appearance directly while viewing the virtual reality from a first-person perspective, but can check the appearance of the changed avatar by looking at the image reflected in the mirror 902. In this case, it is convenient to check the appearance because the mirror appears in the virtual reality without having to search for it.

[0144] Conventionally, in order to replace the avatar image 912 in the profile screen 911, a character model and a clothing model were manually combined, but since the result of the combination by the avatar combination unit can be reflected in the avatar image 912, the profile image can be automatically updated, which is convenient.

[0145] Furthermore, by linking the avatar material sales page with the avatar image 912 in the profile screen 911, the synthesis result by the avatar synthesis unit can be automatically reflected on the avatar material sales page as well as the profile screen 911.

[0146] The present invention is not limited to the above embodiment. For example, the virtual reality provided by the VR providing server 10 can be applied to avatars placed in any virtual reality space, such as a battle game, an event or music festival held in a VR space, an educational facility in a virtual reality space such as a school or a cram school, or a virtual trip.

[0147] In addition, although the above description has been given with three sizes of S, M, and L as examples of clothing models, the sizes are not limited to these three types, and there may be many, for example, 3000 to 5000. Furthermore, the size is merely one example of the expression of the degree of approximation of the clothing model to the body data of the character model. For example, even if "compatible with character model A" or "compatible with character model B" is written instead of the size notation, it is clear that it falls within the technical scope of the present invention by reading it as "corresponding to the size of character model A" or "corresponding to the size of character model B". Even if the size is written using a character model, the sales page control unit 405 may extract a clothing model that matches the body data of the character model according to the degree of approximation of each body data of character model A and character model B. As a result, the present invention can be applied even when the correspondence between the clothing model and the character model is not abstracted as in "S size", but is managed by specifying a specific body such as "compatible with character model A" or "compatible with character model B", or by specifying specific body data such as "common body correspondence between character model A and character model B".

[0148] In the above, the avatar synthesis unit 432 synthesized the character and clothing, which are the avatar materials selected by the VR user terminal 20, but the avatar synthesis unit 432 may also synthesize and display the character and clothing selected by the character model developer terminal 41 or the clothing model developer terminal 42.

[0149] Furthermore, the avatar synthesis unit 432 only needs to synthesize multiple types of character models pre-stored in the VR provision server 10 with a clothing model that is suitable for a specific character model among them, and the character models and clothing models are not limited to those transmitted and obtained from the character model developer terminal 41 and the clothing model developer terminal 42.

[0150] In the above, a target character model is selected, and a sales page with a clothing model that matches the character model and a suggestion for changing clothes (message 824 in FIG. 25) is displayed, but the present invention also includes an embodiment in which a VR user selects a desired clothing model, and the sales page control unit 405 etc. displays a sales page and suggests clothes for the character model that matches the target clothing model, using the selected clothing model as a target clothing model. This embodiment is suitable, for example, when a VR user is looking for a character model that can be dressed in a clothing model that the VR user likes.

[0151] Furthermore, the program according to the present invention is not limited to a single program, but may be a collection of multiple programs. Furthermore, the program according to the present invention is not limited to being executed by a single device, but may be executed by multiple devices in a shared manner. Furthermore, the division of roles between the VR provision server 10 and the VR user terminal 20 is not limited to the above example. That is, part of the processing of the VR provision server 10 may be executed by the VR user terminal 20, and part of the processing of the VR user terminal 20 may be executed by the VR provision server 10.

[0152] Furthermore, a part or all of the means realized by the program can be realized by hardware such as an integrated circuit. Furthermore, the program may be provided by being recorded on a non-transitory recording medium that can be read by a computer. Examples of the recording medium include a hard disk, an SD card, a DVD, and a server on the Internet. [Explanation of symbols]

[0153] 1: Virtual reality provision system, 4: Communication network, 5: Communication network, 10: VR provision server, 11: Processor, 12: RAM, 13: Storage, 13P: Server program, 14: Input / output interface, 15: Communication interface, 18: Display, 19: Communication bus, 20: VR user terminal, 20a: VR user terminal, 20b: VR user terminal, 20c: VR user terminal, 41: Character model developer terminal, 41a: Character model developer terminal, 41b: Character model developer terminal, 42: Clothing model developer terminal, 42a: Clothing Clothing model developer terminal, 42b: clothing model developer terminal, 101: processor, 102: RAM, 103: display, 104: storage, 104P: HMD control program, 105: input interface, 106: motion sensor, 107: rear camera, 108: communication interface, 109: microphone, 110: speaker, 111: communication bus, 200: HMD set, 201: processor, 202: RAM, 203: ROM, 204: operation switch, 205: motion sensor, 206: BT communication device, 209: communication bus, 210: HMD, 211 : HMD main body, 213: HMD side short-range wireless communication device, 214: display, 219: holder, 220: HMD set, 221: remote control side short-range wireless communication device, 222: rear button, 229: housing, 230: remote controller, 240: pointer, 301: processor, 302: RAM, 303: ROM, 304: storage, 304P: terminal program, 305: input interface, 306: communication interface, 309: communication bus, 401: login authentication unit, 402: character model management unit, 403: clothing model management unit, 404: user management unit, 405: sales page control unit, 406: VR drawing data transmission unit, 407: purchase reception unit, 408: payment unit, 409: character model storage unit, 410: clothing model storage unit, 411: VR user data storage unit, 412: developer management data storage unit, 431: VR image generation unit, 432: avatar synthesis unit, 433: avatar movement control unit, 500: developer management table, 520: character model management table, 520a: character model management table, 540: clothing model management table, 540a: clothing model management table, 560: VR user management table,560a: VR user management table, 600: Avatar material sales page, 601: Avatar image, 602: Name, 611: Explanation information, 612: Explanation information, 613: Explanation information, 800: Screen, 802: Purchase screen icon, 810: Screen, 811: Inventory display icon, 812: Menu screen, 820: Screen, 821: Inventory screen, 822: Clothing model, 823: Size selection icon, 824: Advice message, 830: Screen, 831: Purchase decision icon, 832: Purchase completion screen, 833: Icon, 900: Screen, 901: Passage, 902: Mirror, 903: Avatar, 910: Screen, 911: Profile screen, 912: Avatar image, C1 to C12: Tracking point, 801: Avatar, J1 to J12: Joint point (body feature point), M10: Reference point,

Claims

1. A virtual reality display program, comprising: On the computer, A function of making a mirror appear in a virtual reality space when an avatar material used for an avatar operated by a user placed in the virtual reality space is changed; and a function of displaying an avatar image reflecting the avatar on the mirror. Virtual reality display program.

2. 2. The virtual reality providing program according to claim 1, The computer includes: and further realizing a function of replacing the avatar image displayed on the mirror with an avatar image included in the profile of the avatar. Virtual reality display program.

3. 2. The virtual reality providing program according to claim 1, the avatar is a composite avatar obtained by combining a character model including body data having a three-dimensional shape with a clothing model that is adapted to the three-dimensional shape of the body data of the character model, The computer includes: realizing a function of making a mirror appear in the virtual reality space when the clothing model used for the composite avatar is changed; Virtual reality display program.

4. A virtual reality provision system configured by communicatively connecting a virtual reality provision server and a user terminal, The user terminal is an avatar synthesis unit that generates a composite avatar by synthesizing a character model and a clothing model used for an avatar operated by a user that is placed in a virtual reality space; The virtual reality providing server includes: a user management unit that, when the avatar synthesis unit changes the clothing model used for the avatar, makes a mirror appear near a position where the avatar is present in the virtual reality space, and transmits an avatar image in which the avatar is reflected in the mirror to the user terminal; Virtual reality delivery system.

5. 5. The virtual reality providing system according to claim 4, the character model includes body data having a three-dimensional shape defined using body feature points; the clothing model includes skeleton tracking data including tracking points indicating parts of the clothing model that overlap with body feature points of the character model when the clothing model is worn by the character model, the avatar synthesis unit generates the composite avatar by associating body feature points included in the character model with tracking points included in the clothing model; Virtual reality delivery system.

Images