Program and Information Processing Apparatus

Abstract

To provide a program and an information processor that enhances the presence of a virtual space.SOLUTION: A program related to a system for providing a user with an appreciation experience of video and sound in a virtual space causes a computer of an information processor 2 to function as: arrangement means 102 for arranging a plurality of objects including one or more material objects with which material information 142b about a material is associated, one or more light source objects for virtually emitting light in the virtual space, and one or more sound source objects for virtually emitting sound in the virtual space, respectively, in the virtual space which can be recognized by a user using a predetermined terminal device 3; first generation means 104a for generating reflection light information 144a corresponding to the light hitting the material object and echo sound information 144b corresponding to the sound hitting the material object on the basis of the material information; and output means 108 for outputting an image based on the reflection light information and audio based on the echo sound information to the user.SELECTED DRAWING: Figure 1

Description

Technical Field

[0001] The present disclosure relates to a program and an information processing apparatus.

Background Art

[0002] Conventionally, a technique for providing content that realizes an immersive viewing experience has been known. For example, Patent Document 1 discloses a viewing system that provides a viewing user with a viewing experience of content that presents a performer character whose behavior is controlled based on the body movements of the performer. This viewing system controls the behavior of the performer character based on the motion information indicating the body movements of the performer, and presents an effect in a three-dimensional space where the performer character is arranged based on the information of the viewing user's viewpoint.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] However, depending on the viewing system disclosed in Patent Document 1, the sense of presence in the virtual space cannot be sufficiently enhanced.

[0005] Therefore, in view of the above problems, an object of the present disclosure is to enhance the sense of presence in the virtual space.

Means for Solving the Problems

[0006] A program according to one aspect of the present disclosure causes a computer to function as: an arrangement means for arranging a plurality of objects including at least one material object associated with material information regarding a material, a light source object that virtually emits light in a virtual space, and a sound source object that virtually emits sound in the virtual space, in a virtual space recognizable by a user using a predetermined terminal device; a first generation means for generating reflected light information corresponding to light hitting the material object and reverberation sound information corresponding to sound hitting the material object, based on the material information; and an output means for outputting an image based on the reflected light information and a voice based on the reverberation sound information, to the user.

[0007] A program according to another aspect of the present disclosure causes a computer to function as an output means for outputting at least one object including a material object associated with material information regarding a material, a light source object that virtually emits light in a virtual space, and a sound source object that virtually emits sound in the virtual space, in the virtual space so that the user can recognize it, and the output means further outputs reflected light information corresponding to light hitting the material object and reverberation sound information corresponding to sound hitting the material object, generated based on the material information.

[0008] An information processing apparatus according to another aspect of the present disclosure includes: an arrangement unit for arranging at least one object including a material object associated with material information regarding a material, a light source object that virtually emits light in a virtual space, and a sound source object that virtually emits sound in the virtual space, in a virtual space recognizable by a user using a predetermined terminal device; a first generation unit for generating reflected light information corresponding to light hitting the material object and reverberation sound information corresponding to sound hitting the material object, based on the material information; and an output unit for outputting the reflected light information and the reverberation sound information to the user. [Effect of the Invention]

[0009] According to the present disclosure, the sense of presence in the virtual space can be enhanced.

Brief Description of the Drawings

[0010]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Modes for Carrying Out the Invention

[0011] With reference to the accompanying drawings, preferred embodiments of the present disclosure (hereinafter referred to as this embodiment) will be described. In each figure, those with the same reference numerals have the same or similar configurations.

[0012] In the present disclosure, "part", "means", "device" and "system" do not simply mean physical means, but also include cases where the functions of the "part", "means", "device" and "system" are realized by software. Also, even if the functions of one "part", "means", "device" and "system" are realized by two or more physical means and devices, software, or even if the functions of two or more "parts", "means", "devices" and "systems" are realized by one physical means, device, software, it is also acceptable.

[0013] <1. Overview of System 1> System 1 according to this embodiment (hereinafter simply referred to as "System 1") is a system that provides a user with an experience of enjoying images and sounds in a virtual space. The virtual space is a space in virtual reality (VR: Virtual Reality). Virtual reality is a digital environment generated using a computer. An example of the virtual space is the metaverse.

[0014] By using a predetermined terminal device 3, the user can recognize other objects in the virtual space through an object (hereinafter referred to as an "avatar") that serves as the user's alter ego in the virtual space.

[0015] Other objects include, for example, light source objects corresponding to light sources (such as lighting, the sun, and flames, etc.) in the virtual space. By using a head mount display (HDM) or the like as the terminal device 3, the user can visually recognize the light source object as if the user were at the position of the avatar in the virtual space.

[0016] Other objects also include, for example, sound source objects corresponding to sound sources (such as musical instruments, speakers, other users who are speaking, etc.) in the virtual space. By using an HDM or the like as the terminal device 3, the user can hear the sound virtually emitted from the sound source object as if the user were at the position of the avatar in the virtual space.

[0017] The user can also recognize objects other than the light source object and the sound source object (e.g., furniture objects, wall objects, ceiling objects, and avatars of other users, etc.). The light virtually emitted from the light source object (hereinafter referred to as "virtual light") is reflected by such objects. Then, the user can recognize the reflected light due to the reflection (hereinafter referred to as "virtual reflected light"). Also, the sound virtually emitted from the sound source object (hereinafter referred to as "virtual sound") resonates with such objects. Then, the user can recognize the resonant sound due to the resonance (hereinafter referred to as "virtual resonant sound").

[0018] By the way, in the real space, the reflected light and the resonant sound generally vary depending on the material of the object that the light and sound hit (e.g., material, color, texture, density, elasticity, etc.). One of the purposes of System 1 is to reproduce such a phenomenon in the virtual space. Specifically, System 1 determines the virtual reflected light and the virtual resonant sound based on the material information regarding the material associated with the object, and makes them recognizable to the user. Hereinafter, the object associated with the material information is referred to as a "material object".

[0019] <2. Configuration of System 1> With reference to FIG. 1, an example of the configuration of System 1 will be described. System 1 includes an information processing apparatus 2, a terminal apparatus 3, and a communication network 4.

[0020] [Information Processing Apparatus 2] The information processing apparatus 2 includes a control unit 10, a storage unit 14, and a network interface unit 18. The control unit 10, the storage unit 14, and the network interface unit 18 are each electrically connected via a bus 16.

[0021] [Storage Unit 14] The storage unit 14 stores various information necessary for the operation of the information processing apparatus 2. The stored information includes space information 140, object information 142, image information 143, reflected light information 144a, and resonant sound information 144b.

[0022] The spatial information 140 is information regarding the shape of the virtual space and the like. The shape of the virtual space and the like includes information regarding the size of the virtual space (i.e., height, width, depth, etc.). When the virtual space corresponds to the inside of a building, the spatial information 140 includes information regarding the shape, position, etc. of the ceiling, stairs, floor, corridor, and walls of the building. When the virtual space corresponds to the outside, the spatial information 140 includes information regarding the terrain and the like. The spatial information 140 may be three-dimensional structure information such as CAD (Computer Aided Design) data, for example.

[0023] Note that the buildings in the virtual space and the ceiling, stairs, floor, corridor, walls, etc. of the buildings may each be material objects.

[0024] The information processing apparatus 2 can generate a virtual space by reading the spatial information 140. When the information processing apparatus 2 generates a virtual space based on the spatial information 140 including information regarding the shape of a concert hall and the like, for example, the user can experience video and audio as if virtually entering the concert hall via an avatar.

[0025] The object information 142 is information associated with an object, and includes position information 142a, material information 142b, and structure information 142c.

[0026] The position information 142a is information regarding the position of the material object in the virtual space. The position information 142a includes information regarding coordinates on a three-dimensional coordinate axis, for example. The position information 142a may be such that one coordinate (for example, the coordinate corresponding to the center of the material object) is associated with one material object. Also, the position information 142a may be such that a plurality of coordinates (for example, four coordinates at the upper end, lower end, right end, and left end of the material object) are associated with one material object. When the virtual space is generated, the position where the material object is arranged may be determined by reading the position information 142a.

[0027] The material information 142b is information regarding the material of the material object. The user can recognize in the virtual space as if the material object is made of the material related to the material information 142b. For example, when the material related to the material information 142b is "wood", in the virtual space, the material object associated with the material information 142b is recognized as if it is made of "wood". Also, for example, when the material related to the material information 142b is "metal", in the virtual space, the material object associated with the material information 142b is recognized as if it is made of "metal". When the virtual space is generated, the graphics of the material object may be determined by loading the material information 142b.

[0028] The material information 142b may include information regarding the color, texture, density, and elasticity of the material object. The texture of the material object includes the texture and transparency of the material object, etc.

[0029] The texture is the fine pattern or design on the surface of the material object. The texture is, for example, the annual rings of wood, the pattern of stone, and the weave of fabric, etc. When the texture is rough, typically, the material object gives a matte impression. On the other hand, when the texture is smooth, typically, the material object gives an impression of being shiny.

[0030] The transparency is the degree to which the material object transmits virtual light. When the material object is transparent, it gives a smooth and pure impression. On the other hand, when the material object is translucent or opaque, it gives a solid impression.

[0031] The material information 142b may include first material parameters corresponding to the material related to the material information 142b, and one or more second material parameters determined based on the first material parameters. The set values of the first material parameters are, for example, "wood", "metal", "rubber", "plastic", "foaming agent", etc. The second material parameters are, for example, color, texture, density, elasticity, etc. determined based on the first material parameters.

[0032] The second material parameters include reflection-related parameters related to the generation of the reflected light information 144a and reverberation-related parameters related to the generation of the reverberation sound information 144b. The reflection-related parameters are, for example, the color and texture of the material object, etc. The reverberation-related parameters are, for example, the density and elasticity of the material object, etc.

[0033] The structure information 142c includes information related to the structure of the material object. The structure of the material object includes, for example, the height, width, depth (thickness) of the material object, and internal cavities, etc. The structure information 142c may be three-dimensional structure information such as CAD data. When the virtual space is generated, the graphic of the material object may be determined by reading the structure information 142c.

[0034] The information processing device 2 generates a virtual space in which the object is arranged based on the space information 140 and the object information 142.

[0035] The image information 143 is information generated by photographing an object in the real space. The image information 143 may be stored in a JPEG (Joint Photographic Experts Group) file or the like.

[0036] The reflected light information 144a is information related to the virtual reflected light when the virtual light emitted from the light source object hits the material object. The generation of the reflected light information 144a will be described later with reference to FIG. 7.

[0037] The reverberation sound information 144b is information regarding the virtual reverberation sound when the virtual sound emitted from the sound source object hits the material object. The generation of the reverberation sound information 144b will be described later with reference to FIG. 7.

[0038] [Control Unit 10] The control unit 10 functions as a reception unit 100, an arrangement unit 102, a generation unit 104, an acquisition unit 106, and an output unit 108 by executing various programs stored in the storage unit 14. Hereinafter, each function will be described.

[0039] - Reception Unit 100 - The reception unit 100 includes a first reception unit 100a, a second reception unit 100b, and a third reception unit 100c.

[0040] The first reception unit 100a receives a first operation regarding the arrangement of at least one of a plurality of objects including a material object, a light source object, and a sound source object from the user. The first operation may be performed in the virtual space (for example, via a three-dimensional graphical user interface visible to the user), or may be performed in the real space (for example, via a user interface for selecting a position on a screen on which a plan view or a sketch of the virtual space is displayed). The first operation may be the specification of the position of the object. Also, the first operation may be to read an image in which an object is arranged in the real space.

[0041] The second reception unit 100b receives a second operation regarding the change of the material information 142b from the user. The second operation is typically an operation regarding the change of the first material parameter. The operation regarding the change of the first material parameter is, for example, an operation of changing the material of the material object from "wood" to "metal". In this case, in response to the change of the first material parameter, the second material parameter of the material object may be changed from the color, texture, density, elasticity, etc. corresponding to "wood" to the color, texture, density, elasticity, etc. corresponding to "metal".

[0042] Note that the second material parameter may not be directly modifiable by the user. The second material parameter may be changed dependently when the first material parameter is changed. That is, the second reception means 100b receives an operation related to the change of the first material parameter, and the second material parameter may be changed according to the change of the first material parameter.

[0043] The third reception means 100c receives from the user a third operation related to the change of at least one of the light source setting information associated with the light source object and the sound source setting information associated with the sound source object.

[0044] The light source object emits virtual light based on the light source setting information. The light source object may emit first virtual light when the light source setting information is set to a certain value, and may emit second virtual light when the light source setting information is set to another value. The virtual reflected light is determined based on the virtual light and the material information 142b of the material object that the virtual light hits. That is, when the light source setting information is changed, the virtual light changes, and accordingly the virtual reflected light also changes. The light source setting information includes, for example, information regarding the type of light source (e.g., LED (light-emitting diode), fluorescent lamp, incandescent lamp, halogen lamp, HID (High Intensity Discharge Lamp), and ozone lamp, etc.), color temperature, luminous flux, illuminance, shape of the light source, position of the light source, and direction in which the light is emitted.

[0045] The sound source object emits virtual sound based on the sound source setting information. The sound source object may emit first virtual sound when the sound source setting information is set to a certain value, and may emit second virtual sound when the sound source setting information is set to another value. The virtual reverberant sound is determined based on the virtual sound and the material information 142b of the material object that the virtual sound hits. That is, when the sound source setting information is changed, the virtual sound changes, and accordingly the virtual reverberant sound also changes. The sound source setting information includes, for example, information regarding volume, pitch, timbre, piece of music, shape of the sound source, position of the sound source, and direction in which the sound is emitted.

[0046] - Arrangement means 102 - The arrangement means 102 arranges a plurality of objects each including one or more material objects, light source objects, and sound source objects in the virtual space. The arrangement means 102 may arrange at least one of the plurality of objects based on the first operation received by the first reception means 100a. For example, when the first operation is an operation related to specifying the position of an object, the arrangement means 102 arranges the object at the position. Additionally, for example, when the first operation is an operation for causing an image in which an object is arranged in the real space to be read, the arrangement means 102 identifies the position of the object in the real space by analyzing the image, and arranges an object corresponding to the object at the position in the virtual space corresponding to the position.

[0047] - Generation means 104 - The generation means 104 includes a first generation means 104a and a second generation means 104b.

[0048] The first generation means 104a generates reflected light information 144a corresponding to virtual light hitting the material object and reverberation sound information 144b corresponding to virtual sound hitting the material object based on the material information 142b. The reflected light information 144a and the reverberation sound information 144b may be determined based on a second material parameter determined based on the first material parameter. Specifically, the reflected light information 144a may be generated based on a reflection-related parameter (for example, the color and texture of the material object, etc.) among the second material parameters, and the reverberation sound information 144b may be generated based on a reverberation-related parameter (for example, the density and elasticity of the material object, etc.) among the second material parameters.

[0049] Also, the first generation means 104a may generate the reflected light information 144a based on the material information 142b, and may generate the reverberation sound information 144b based on the structure information 142c and the material information 142b. With reference to FIGS. 2 - 3, an example of the reflected light information 144a and the reverberation sound information 144b generated in this way will be described.

[0050] FIG. 2 is a diagram showing an example of virtual reflected light when the same virtual light A1 hits from the same direction each of two material objects, i.e., material object 208a and material object 208b, which share material information 142b but do not share structure information 142c (in this example, thickness). Let the virtual reflected light when virtual light A1 hits material object 208a be virtual reflected light A'1, and the virtual reflected light when virtual light A1 hits material object 208b be virtual reflected light A'2. At this time, since the reflected light information 144a is generated based on the material information 142b (regardless of the structure information 142c), the virtual reflected light A'1 and the virtual reflected light A'2 are substantially the same.

[0051] FIG. 3 is a diagram showing an example of virtual reverberant sound when the same virtual sound B1 hits from the same direction each of material object 208a and material object 208b. Let the virtual reverberant sound when virtual sound B1 hits material object 208a be virtual reverberant sound B'1, and the virtual reverberant sound when virtual sound B1 hits material object 208b be virtual reverberant sound B'2. At this time, since the reverberant sound information 144b is generated based on not only the material information 142b but also the structure information 142c that is not common between the two material objects, the virtual reverberant sound B'1 and the virtual reverberant sound B'2 are different.

[0052] Also, when the material object includes a first material object and a second material object provided on the surface of the first material object, and the material information 142b includes first material information associated with the first material object and second material information associated with the second material object, the first generation means 104a may generate the reflected light information 144a based on the second material information, and may generate the reverberant sound information 144b based on the first material information and the second material information. An example of the reflected light information 144a and the reverberant sound information 144b generated in this way will be described with reference to FIGS. 4 - 5.

[0053] FIG. 4 is a diagram showing an example of virtual reflected light when the same virtual light C1 hits each of the material objects 208c and 208d from the same direction. The material object 208c is composed of a first material object 210b and a second material object 210a provided on the surface of the first material object 210b. The material object 208d is composed of a first material object 210d and a second material object 210a provided on the surface of the first material object 210b. That is, the material objects 208c and 208d share the second material object 210a provided on the surface, and the first material objects 210b / 20d inside them do not share.

[0054] When the virtual light C1 hits the surface of the material object 208c where the second material object 210a is provided, the virtual reflected light is defined as virtual reflected light C’1. When the virtual light C1 hits the surface of the material object 208d where the second material object 210a is provided, the virtual reflected light is defined as virtual reflected light C’2. At this time, since the reflected light information 144a is generated based on the second material information associated with the second material object 210a (regardless of the first material information associated with the first material object 210b / 210d), the virtual reflected lights A’1 and A’2 are substantially the same.

[0055] FIG. 5 is a diagram showing an example of virtual reverberation sound when the same virtual sound D1 hits each of the material objects 208c and 208d from the same direction. When the virtual sound D1 hits the surface of the material object 208c where the second material object 210a is provided, the virtual reverberation sound is defined as virtual reverberation sound D’1. When the virtual sound D1 hits the surface of the material object 208d where the second material object 210a is provided, the virtual reverberation sound is defined as virtual reverberation sound D’2. At this time, since the reverberation sound information 144b is generated based on the first material information associated with the first material object 210b / 210d in addition to the second material information associated with the second material object 210a, the virtual reverberation sounds D’1 and D’2 are different.

[0056] The second generation means 104b generates the material information 142b according to the material of the object identified based on the image information 143 regarding the object in the real space. For example, when the acquisition means 106 described later acquires the image information 143 regarding the image of an object made of "wood", the second generation means 104b determines the second material parameters (for example, the color and texture of "wood" shown in the image) based on the image information 143 through image recognition processing or the like, and may generate the material information 142b having such second material parameters.

[0057] ―Acquisition means 106― The acquisition means 106 acquires the image information 143. Note that acquiring information includes making the information in a state processable by the control unit 10. That is, the acquisition means 106 may receive the image information 143 from another device, or may read out the image information 143 stored in the storage unit 14.

[0058] ―Output means 108― The output means 108 outputs an image (including video) based on the reflected light information 144a and voice based on the reverberation sound information 144b to the user. The output means 108 may output an image based on the reflected light information 144a by, for example, functioning the display of the HMD worn by the user, and may output voice based on the reverberation sound information 144b by functioning the speaker.

[0059] [Terminal device 3] The terminal device 3 is a terminal operated by the user. The terminal device 3 includes a control unit 20, a storage unit (not shown), and a network interface unit (not shown).

[0060] The control unit 20 functions as a reception means 200, an acquisition means 202, and an output means 204 by executing various programs stored in the storage unit of the terminal device 3. Each function will be described below.

[0061] ―Reception means 200― The reception means 200 receives from the user a first operation regarding the arrangement of one or more objects including a material object, a light source object, and a sound source object, a second operation regarding the change of the material information 142b, and a third operation regarding the change of at least one of the light source setting information and the sound source setting information. Information regarding the first to third operations is transmitted to the information processing apparatus 2 via the network interface unit of the terminal device 3.

[0062] ― Acquisition means 202 ― The acquisition means 202 acquires from the information processing apparatus 2 information for outputting an image based on the reflected light information 144a and sound for outputting a voice based on the reverberation sound information 144b.

[0063] ― Output means 204 ― The output means 204 outputs one or more objects including a material object, a light source object, and a sound source object in the virtual space so that the user can recognize them. Further, the output means 204 outputs the reflected light information 144a and the reverberation sound information 144b generated based on the material information 142b.

[0064] <3. Operation of the information processing apparatus 2> With reference to FIGS. 6 - 7, an example of the operation of the information processing apparatus 2 will be described. FIG. 6 is a flowchart for explaining an example of the operation of the information processing apparatus 2. FIG. 7 is a conceptual diagram of a virtual space in which objects and an avatar exist.

[0065] In the initial state, it is assumed that the user's avatar is located at the position "L1" in the virtual space (see the center of FIG. 7).

[0066] First, the information processing apparatus 2 receives a designation of the position of the light source object from the user (S10). In this example, it is assumed that the user designates "L2" as the position of the light source object (see the upper left side of FIG. 7). The information processing apparatus 2 arranges the light source object based on the user's operation. Virtual light E1 is emitted from the light source object.

[0067] Next, the information processing apparatus 2 receives a designation of the position of the sound source object from the user (S12). In this example, it is assumed that the user designates "L3" as the position of the sound source object (see the upper right side of FIG. 7). The information processing apparatus 2 arranges the sound source object based on the user's operation. A virtual sound F1 is emitted from the sound source object.

[0068] Next, the information processing apparatus 2 receives a designation of the position of the material object from the user (S14). In this example, it is assumed that the user designates "L4" as the position of the material object (see the lower side of FIG. 7). The information processing apparatus 2 arranges the material object based on the user's operation.

[0069] Next, the information processing apparatus 2 generates reflected light information 144a based on the positional relationship among the avatar, the light source object, and the material object, and the material information 142b associated with the material object (S16). The path r1 in FIG. 7 is an example of a path in which the virtual light E1 emitted from the light source object hits the material object and reaches the avatar. Virtual reflected light E'1 reaches the avatar. The information processing apparatus 2 generates the reflected light information 144a according to the virtual reflected light E'1.

[0070] Next, the information processing apparatus 2 generates stereoscopic video information based on the generated reflected light information 144a (S18).

[0071] Next, the information processing apparatus 2 generates reverberation sound information 144b based on the positional relationship among the avatar, the sound source object, and the material object, and the material information 142b associated with the material object (S20). The path r2 in FIG. 7 is an example of a path in which the virtual sound F1 emitted from the sound source object hits the material object and reaches the avatar. Virtual reverberation sound F'1 reaches the avatar. The information processing apparatus 2 generates the reverberation sound information 144b according to the virtual reverberation sound F'1.

[0072] Next, the information processing apparatus 2 generates stereophonic audio information based on the generated reverberation sound information 144b (S22).

[0073] Next, the information processing apparatus 2 outputs the stereoscopic video information generated in S18 and the stereoscopic audio information generated in S22 (S24). As a result, the user can recognize the virtual reflected light E'1 reflected on the material object and the virtual reverberant sound F'1 reverberated on the material object.

[0074] To summarize the above, the program according to an embodiment of the present disclosure causes the computer 70 of the information processing apparatus 2 to include in a virtual space recognizable by the user using the predetermined terminal device 3 a material object associated with material information 142b regarding a material, a light source object that virtually emits light in the virtual space, and a sound source object that virtually emits sound in the virtual space, respectively, an arrangement unit 102 that arranges a plurality of objects each including one or more thereof, a first generation unit 104a that generates reflected light information 144a corresponding to the light hitting the material object and reverberant sound information 144b corresponding to the sound hitting the material object based on the material information 142b, and an output unit 108 that outputs an image based on the reflected light information 144a and sound based on the reverberant sound information 144b to the user, and functions as such.

[0075] <4. Effects of the information processing apparatus 2> The information processing apparatus 2 can enhance the sense of presence in the virtual space. With the information processing apparatus 2, the user can recognize virtual reflected light and virtual reverberant sound corresponding to the material of the material object in the virtual space. For example, from a material object of "wood", the user can receive a visual impression as if it is warm and hear a soft and low reverberant sound. In addition, for example, from a material object of "metal", the user can receive a visual impression as if it is cold and hear a hard and high reverberant sound.

[0076] In order to enhance the sense of presence in the virtual space, it is important to enable the user to experience vision and hearing integrally. According to the information processing apparatus 2, since the user receives a consistent visual impression and auditory impression with respect to the material object, the sense of presence in the virtual space can be enhanced.

[0077] Also, the program according to an embodiment of the present disclosure further functions to cause the computer 70 of the information processing apparatus 2 to function as a first reception unit 100a that receives a first operation regarding the arrangement of at least one of a plurality of objects from a user, and the arrangement unit 102 arranges at least one of the plurality of objects based on the first operation.

[0078] With this configuration, the information processing apparatus 2 can improve the efficiency of designing (or designing) the virtual space. As described with reference to FIG. 7, the virtual reflected light and virtual reverberation sound recognized by the user are determined based on the positional relationship between the user's avatar, the light source object, the sound source object, and the material object. In particular, since the position of the material object is related to both the virtual reflected light and the virtual reverberation sound, it has a great influence on the impression received by the user. That is, with the above configuration, the information processing apparatus 2 can create a virtual space that gives various impressions according to the user's intention.

[0079] Also, the program according to an embodiment of the present disclosure further functions to cause the computer 70 of the information processing apparatus 2 to function as a second reception unit 100b that receives a second operation regarding the change of the material information 142b from the user.

[0080] With this configuration, the information processing apparatus 2 can improve the efficiency of designing (or designing) the virtual space. As described with reference to FIG. 7, the virtual reflected light and virtual reverberation sound recognized by the user are determined based on the material information 142b associated with the material object. That is, when the material information 142b is changed, both the virtual reflected light and the virtual reverberation sound change. Thereby, with the above configuration, the user can more efficiently design a virtual space that gives a consistent impression.

[0081] Also, in the program according to an embodiment of the present disclosure, the virtual light corresponding to the reflected light information 144a is emitted based on the light source setting information associated with the light source object, the virtual sound corresponding to the reverberation sound information 144b is emitted based on the sound source setting information associated with the sound source object, and the program further functions as a third reception means 100c for receiving from the user a third operation regarding at least one of the light source setting information and the sound source setting information by the computer 70 of the information processing apparatus 2.

[0082] With this configuration, the information processing apparatus 2 can streamline the design of the virtual space. As described with reference to FIG. 7, the virtual reflected light and the virtual reverberation sound recognized by the user are determined based on the virtual light emitted from the light source object and the virtual sound emitted from the sound source object, respectively. That is, the light source setting information regarding the setting of the virtual light and the sound source setting information regarding the setting of the virtual sound greatly affect the impression received by the user. With the above configuration, the information processing apparatus 2 can create a virtual space that gives various impressions according to the user's intention.

[0083] Also, in the program according to an embodiment of the present disclosure, the material object is further associated with structure information 142c regarding the structure of the material object, the reflected light information 144a is generated based on the material information 142b, and the reverberation sound information 144b is generated based on the structure information 142c and the material information 142b.

[0084] With this configuration, the information processing apparatus 2 can further enhance the sense of presence in the virtual space. In the real space, the reflected light is typically determined by the material of the object (including texture). On the other hand, the reverberant sound varies depending on the structure of the object (e.g., the thickness of the object and internal cavities, etc.) in addition to the material of the object. According to the above configuration, such physical phenomena in the real space can be reproduced in the virtual space. Specifically, the information processing apparatus 2 determines the virtual reflected light according to the material of the material object, and determines the virtual reverberant sound according to the material of the material object and its structure. Thereby, the information processing apparatus 2 can give the user an impression closer to the real space in the virtual space.

[0085] Also, in the program according to an embodiment of the present disclosure, the material object includes a first material object and a second material object provided on the surface of the first material object, the material information 142b includes first material information associated with the first material object and second material information associated with the second material object, the reflected light information 144a is generated based on the second material information, and the reverberant sound information 144b is generated based on the first material information and the second material information.

[0086] With this configuration, the information processing apparatus 2 can further enhance the sense of presence in the virtual space. In the real space, the reflected light is typically determined by the material of the surface of the object. On the other hand, the reverberant sound varies depending on the material inside the object in addition to the material of the surface of the object. According to the above configuration, such physical phenomena in the real space can be reproduced in the virtual space. Specifically, the information processing apparatus 2 determines the virtual reflected light according to the material of the surface of the material object, and determines the virtual reverberant sound according to the material of the surface of the material object and the material inside it. Thereby, the information processing apparatus 2 can give the user an impression closer to the real space in the virtual space.

[0087] Also, a program according to an embodiment of the present disclosure further functions as an acquisition unit 106 that acquires image information 143 regarding an object in the real space, and a second generation unit 104b that generates material information 142b according to the material of the object specified based on the image information 143, for a computer 70.

[0088] With this configuration, the information processing apparatus 2 can improve the efficiency of designing the virtual space. For example, a technology for reproducing the real space in the virtual space, such as digital twin technology, is known. According to the above configuration, since the material information 142b corresponding to the material of the object in the real space can be easily generated, the virtual space can be designed more efficiently.

[0089] <5. Hardware Configuration> With reference to FIG. 8, an example of the hardware configuration when the information processing apparatus 2 described above is realized by a computer 70 will be described. Note that the functions of each device can also be realized by dividing them among a plurality of devices.

[0090] As shown in FIG. 8, the computer 70 includes a processor 700, a storage device 702, an input I / F 704, a data I / F 706, a communication I / F 708, and a display device 710.

[0091] The processor 700 controls various processes in the computer 70 by executing a program stored in the storage device 702. For example, each functional unit included in the control unit 10 of the information processing apparatus 2 can be realized by the processor 700 executing a program stored in the storage device 702.

[0092] The storage device 702 is a storage medium such as a RAM (Random Access Memory), for example. The RAM temporarily stores the program code of the program executed by the processor 700 and the data required during the execution of the program.

[0093] The memory device 702 is also a non-volatile memory medium such as, for example, a hard disk drive (HDD) and a flash memory. The memory device 702 stores an operating system and various programs for implementing each of the above configurations. The memory medium storing the various programs may be a non-transitory computer readable medium. In addition, the memory device 702 can also store a table for registering various information and a DB for managing the table. Such programs and data are referred to from the processor 700 by being loaded into the memory device 702 as necessary.

[0094] The input I / F 704 is a device for receiving an input from a user. Specific examples of the input I / F 704 include a camera, a button, a microphone, a keyboard, a mouse, a touch panel, various sensors, a wearable device, and the like. The input I / F 704 may be connected to the computer 70 via an interface such as, for example, USB (Universal Serial Bus).

[0095] The data I / F 706 is a device for inputting data from outside the computer 70. Specific examples of the data I / F 706 include a drive device for reading data stored in various memory media. The data I / F 706 may be provided outside the computer 70. In that case, the data I / F 706 is connected to the computer 70 via an interface such as, for example, USB.

[0096] The communication I / F 708 is a device for performing data communication via the communication network 4 with a device outside the computer 70 by wire or wirelessly. The communication I / F 708 may be provided outside the computer 70. In that case, the communication I / F 708 is connected to the computer 70 via an interface such as, for example, USB.

[0097] The display device 710 is a device for displaying various types of information. Specific examples of the display device 710 include, for example, a liquid crystal display, an organic EL (Electro-Luminescence) display, and a display of a wearable device. The display device 710 may be provided outside the computer 70. In that case, the display device 710 is connected to the computer 70 via, for example, a display cable. Also, when a touch panel is adopted as the input I / F 704, the display device 710 can be configured integrally with the input I / F 704.

[0098] Moreover, it is assumed that the components included in the information processing apparatus 2 described in the above embodiment are realized by the program stored in the storage device 702 being executed by the processor 700 and the determined processing cooperating with other hardware. In other words, these components are assumed as software or firmware as well as the corresponding hardware, and in both concepts, they are also described as "function", "means", "section", "processing circuit", "unit", or "module", etc., and can be read as such respectively.

[0099] <6. Other Embodiments> The following aspects are also included in the scope of the present disclosure.

[0100] [Appendix 1] A program according to an embodiment of the present disclosure causes a computer 70 to include in a virtual space recognizable by a user using a predetermined terminal device 3, a material object associated with material information 142b regarding a material, a light source object that virtually emits light in the virtual space, and a sound source object that virtually emits sound in the virtual space, an arranging unit 102 that arranges a plurality of objects each including one or more of them, a first generation unit 104a that generates reflected light information 144a corresponding to light hitting the material object and reverberant sound information 144b corresponding to sound hitting the material object based on the material information 142b, and an output unit 108 that outputs an image based on the reflected light information 144a and voice based on the reverberant sound information 144b to the user, and functions as such.

[0101] [Appendix 2] The program described in Appendix 1 may further cause the computer 70 to function as a first reception unit 100a that receives from the user a first operation regarding the arrangement of at least one of the plurality of objects, and the arranging unit 102 may arrange at least one of the plurality of objects based on the first operation.

[0102] [Appendix 3] The program described in Appendix 1 or Appendix 2 may further cause the computer 70 to function as a second reception unit 100b that receives from the user a second operation regarding the change of the material information 142b.

[0103] [Appendix 4] In the program described in any one of Appendices 1 to 3, the light corresponding to the reflected light information 144a may be virtually emitted based on light source setting information associated with the light source object, the sound corresponding to the reverberant sound information 144b may be virtually emitted based on sound source setting information associated with the sound source object, and the program may further cause the computer 70 to function as a third reception unit 100c that receives from the user a third operation regarding the change of at least one of the light source setting information and the sound source setting information.

[0104] [Appendix 5] In the program according to any one of Supplementary Notes 1 to 4, the material object may further be associated with structure information 142c regarding the structure of the material object, the reflected light information 144a may be generated based on the material information 142b, and the reverberation sound information 144b may be generated based on the structure information 142c and the material information 142b.

[0105] [Supplementary Note 6] In the program according to Supplementary Note 5, the structure information 142c may include information regarding at least one of the thickness and the cavity of the material object.

[0106] [Supplementary Note 7] In the program according to any one of Supplementary Notes 1 to 6, the material object may include a first material object and a second material object provided on the surface of the first material object, the material information 142b may include first material information associated with the first material object and second material information associated with the second material object, the reflected light information 144a may be generated based on the second material information, and the reverberation sound information 144b may be generated based on the first material information and the second material information.

[0107] [Supplementary Note 8] The program according to any one of Supplementary Notes 1 to 7 may further function as an acquisition means 106 for acquiring image information 143 regarding an object in the real space and a second generation means 104b for generating the material information 142b according to the material of the object specified based on the image information 143 in a computer 70.

[0108] [Supplementary Note 9] In the program according to any one of Supplementary Notes 1 to 8, the material information 142b may include information regarding at least a part of the color, texture, density, and elasticity of the material object.

[0109] [Supplementary Note 10] A program according to another embodiment of the present disclosure causes a computer 70 to function as output means 204 that outputs one or more objects including a material object associated with material information 142b regarding a material, a light source object that virtually emits light in a virtual space, and a sound source object that virtually emits sound in the virtual space so that a user can recognize them. The output means 204 further outputs reflected light information 144a corresponding to light hitting the material object and reverberation sound information 144b corresponding to sound hitting the material object, which are generated based on the material information 142b.

[0110] [Appendix 11] An information processing apparatus 2 according to another embodiment of the present disclosure includes an arrangement unit that arranges one or more objects including a material object associated with material information 142b regarding a material, a light source object that virtually emits light in a virtual space, and a sound source object that virtually emits sound in the virtual space in a virtual space that can be recognized by a user using a predetermined terminal device 3, a first generation unit that generates reflected light information 144a corresponding to light hitting the material object and reverberation sound information 144b corresponding to sound hitting the material object based on the material information 142b, and an output unit that outputs the reflected light information 144a and the reverberation sound information 144b to the user.

Explanation of Reference Numerals

[0111] 2... Information processing apparatus, 3... Terminal device, 70... Computer, 100... Reception means, 100a... First reception means, 100b... Second reception means, 100c... Third reception means, 102... Arrangement means, 104... Generation means, 104a... First generation means, 104b... Second generation means, 106... Acquisition means, 108... Output means, 142b... Material information, 142c... Structure information, 143... Image information, 144a... Reflected light information, 144b... Reverberation sound information, 200... Reception means, 202... Acquisition means, 204... Output means

Claims

1. A computer, in a virtual space recognizable by a user using a predetermined terminal device, an object material related to material information associated with the material, a light source object that virtually emits light in the virtual space, and a sound source object that virtually emits sound in the virtual space, arranging means for arranging a plurality of objects each including one or more of them, first generation means for generating reflected light information corresponding to the light hitting the material object and reverberation sound information corresponding to the sound hitting the material object based on the material information, output means for outputting an image based on the reflected light information and a voice based on the reverberation sound information to the user, functioning as, the material object is further associated with structure information regarding the structure of the material object, the reverberation sound information is generated further based on the structure information, a program.

2. A computer, in a virtual space recognizable by a user using a predetermined terminal device, an object material related to material information associated with the material, a light source object that virtually emits light in the virtual space, and a sound source object that virtually emits sound in the virtual space, arranging means for arranging a plurality of objects each including one or more of them, first generation means for generating reflected light information corresponding to the light hitting the material object and reverberation sound information corresponding to the sound hitting the material object based on the material information, output means for outputting an image based on the reflected light information and a voice based on the reverberation sound information to the user, functioning as, the material object includes a first material object and a second material object provided on the surface of the first material object, the material information includes first material information associated with the first material object and second material information associated with the second material object, the reflected light information is generated based on the second material information, the reverberation sound information is generated based on the first material information and the second material information, a program.

3. further functioning the computer as first reception means for receiving a first operation regarding the arrangement of at least one of the plurality of objects from the user, The program according to claim 1 or 2, wherein the arranging means arranges at least one of the plurality of objects based on the first operation.

4. The program according to claim 1 or 2, further causing the computer to function as second reception means for receiving from the user a second operation regarding a change in the material information.

5. The light is virtually emitted based on light source setting information associated with the light source object. The sound is virtually emitted based on sound source setting information associated with the sound source object. The program according to claim 1 or 2, further causing the computer to function as third reception means for receiving from the user a third operation regarding a change in at least one of the light source setting information and the sound source setting information.

6. The program according to claim 1, wherein the structure information includes information regarding at least one of the thickness and the cavity of the material object.

7. The computer acquisition means for acquiring image information regarding an object in the real space; second generation means for generating the material information according to the material of the object identified based on the image information; The program according to claim 1 or 2, further causing the computer to function as such.

8. The program according to claim 1 or 2, wherein the material information includes information regarding at least a part of the color, texture, density, and elasticity of the material object.

9. The computer functions as output means for outputting one or more objects including a material object associated with material information regarding a material in a virtual space, a light source object that virtually emits light in the virtual space, and a sound source object that virtually emits sound in the virtual space so that the user can recognize them. The output means further outputs reflection light information corresponding to the light hitting the material object and reverberation sound information corresponding to the sound hitting the material object, both generated based on the material information. The material object is further associated with structure information regarding the structure of the material object. The reverberation sound information is generated further based on the structure information.

10. The computer Function as output means for outputting one or more objects including a material object associated with material information regarding a material in a virtual space, a light source object that virtually emits light in the virtual space, and a sound source object that virtually emits sound in the virtual space so that they can be recognized by a user. The output means further outputs reflection light information corresponding to the light hitting the material object and reverberation sound information corresponding to the sound hitting the material object, both of which are generated based on the material information. The material object includes a first material object and a second material object provided on the surface of the first material object. The material information includes first material information associated with the first material object and second material information associated with the second material object. The reflection light information is generated based on the second material information. The reverberation sound information is generated based on the first material information and the second material information. A program.

11. A storage unit that stores the program according to any one of Claims 1, 2, 9, and 10, A control unit that executes the program, An information processing apparatus comprising the above.

Images