Video-playing method and device for 3D virtual space using indirect lighting

Abstract

An embodiment of the present invention relates to a video-playing method and device for a 3D virtual space using indirect lighting. The video-playing method and device for a 3D virtual space using indirect lighting can create lighting effect when a video plays in a virtual space, and thus provide an immersive video-viewing experience.

Description

Method and apparatus for playing 3D virtual space video using indirect lighting

[0001] The present invention relates to a method and apparatus for playing a three-dimensional virtual space video using indirect lighting, and more specifically, to a method and apparatus for playing a three-dimensional virtual space video using indirect lighting that generates a lighting effect according to the video when playing a video in a virtual space.

[0002] With the rapid recent advancement of Virtual Reality (VR) technology, users have been able to realistically experience three-dimensional spaces. To achieve this, various graphics technologies, such as 3D modeling, animation, texture mapping, and lighting effects, are utilized comprehensively. In particular, computer graphics APIs are widely used, enabling real-time rendering through GPUs (Graphics Processing Units).

[0003] VR applications developed using these computer graphics APIs have enhanced realism by applying Physically Based Rendering (PBR) techniques to closely replicate actual light reflection characteristics. As a result, it has become possible to display realistic models and spaces in real-time within a VR environment, and their utility is increasing in various industrial fields (games, entertainment, education, etc.).

[0004] Due to the development of XR (eXtended Reality) technologies, such as VR (Virtual Reality), AR (Augmented Reality), MR (Mixed Reality), and holograms, which expand experiences in virtual space and increase immersion, various technologies are being developed and applied to perform and watch performances by artists in virtual reality.

[0005] However, regarding current video playback in virtual space, there is a problem in that there are no methods to further engage users or play videos in the way desired by artists or users, beyond simply playing the video.

[0006] The method and device for playing a three-dimensional virtual space video using indirect lighting according to an embodiment of the present invention are intended to generate lighting effects when playing a video in a virtual space.

[0007] In addition, the method and device for playing a three-dimensional virtual space video using indirect lighting according to an embodiment of the present invention are intended to provide immersive video viewing in a virtual space.

[0008] However, the technical problem that this embodiment aims to solve is not limited to the technical problem described above, and other technical problems may exist.

[0009] As a technical means for achieving the technical problem described above, a method for playing a 3D virtual space video using indirect lighting according to an embodiment of the present invention comprises the steps of: creating a virtual space in the form of a 3D modeled room; creating a video playback object for video playback and arranging the video playback object at a predetermined interval from the wall surface of the virtual space; creating a plurality of lighting objects to generate a lighting effect between the wall surface of the virtual space and the video playback object; setting a user's viewpoint in the center of the virtual space; performing texture rendering for the video to be played; and generating a lighting effect of a plurality of lighting objects based on color information of the video played in the video playback object.

[0010] In addition, a three-dimensional virtual space video playback device utilizing indirect lighting according to an embodiment of the present invention includes a communication module that performs information transmission and reception between a terminal and a virtual space video playback management server, a memory that stores at least one instruction, and a processor that executes at least one instruction. The processor creates a virtual space in the form of a three-dimensionally modeled room, creates a video playback object for video playback, arranges the video playback object to have a preset interval on the wall surface of the virtual space, creates a plurality of lighting objects to generate a lighting effect between the wall surface of the virtual space and the video playback object, sets a user's viewpoint in the center of the virtual space, performs texture rendering for the video to be played, and generates a lighting effect of the plurality of lighting objects based on color information of the video played on the video playback object.

[0011] A method and device for playing a three-dimensional virtual space video using indirect lighting according to an embodiment of the present invention can generate lighting effects when playing a video in a virtual space.

[0012] In addition, the method and device for playing three-dimensional virtual space video using indirect lighting according to an embodiment of the present invention can provide immersive video viewing in a virtual space.

[0013] FIG. 1 is a conceptual diagram illustrating a method for playing a three-dimensional virtual space video using indirect lighting according to an embodiment of the present invention.

[0014] FIG. 2 is an example diagram showing a communication connection of a three-dimensional virtual space video playback device utilizing indirect lighting according to an embodiment of the present invention.

[0015] FIG. 3 is a configuration diagram of a 3D virtual space video playback management server utilizing indirect lighting according to an embodiment of the present invention.

[0016] FIG. 4 is a configuration diagram of a terminal according to an embodiment of the present invention.

[0017] FIG. 5 is a conceptual diagram illustrating the functions of a 3D virtual space video playback management server utilizing indirect lighting according to an embodiment of the present invention.

[0018] FIG. 6 is a conceptual diagram illustrating the functions of a terminal according to an embodiment of the present invention.

[0019] FIG. 7 is a flowchart of a method for playing a 3D virtual space video using indirect lighting according to an embodiment of the present invention.

[0020] Embodiments of the present invention are described below with reference to the attached drawings so that those skilled in the art can easily implement the invention. However, the present invention may be embodied in various different forms and is not limited to the embodiments described herein. Furthermore, in order to clearly explain the present invention in the drawings, parts unrelated to the explanation have been omitted, and similar parts throughout the specification are denoted by similar reference numerals.

[0021] Throughout the specification, when a part is described as being "connected" to another part, this includes not only cases where they are "directly connected," but also cases where they are "electrically connected" with other components interposed between them. Furthermore, when a part is described as "including" a certain component, this means that, unless specifically stated otherwise, it does not exclude other components but may include additional components.

[0022] In addition, the attached drawings are intended only to facilitate understanding of the embodiments disclosed in this specification, and the technical concept disclosed in this specification is not limited by the attached drawings; it should be understood that all modifications, equivalents, and substitutions included within the concept and technical scope of the present invention are included.

[0023] Terms including ordinal numbers, such as first, second, etc., may be used to describe various components, but said components are not limited by said terms. These terms are used solely for the purpose of distinguishing one component from another.

[0024] When it is stated that one component is "connected" or "connected" to another component, it should be understood that while it may be directly connected or connected to that other component, there may also be other components in between. On the other hand, when it is stated that one component is "directly connected" or "directly connected" to another component, it should be understood that there are no other components in between.

[0025] A singular expression includes a plural expression unless the context clearly indicates otherwise.

[0026] In this application, terms such as “comprising” or “having” are intended to specify the existence of the features, numbers, steps, actions, components, parts, or combinations thereof described in the specification, and should be understood as not precluding the existence or addition of one or more other features, numbers, steps, actions, components, parts, or combinations thereof.

[0027] Hereinafter, a method for playing a three-dimensional virtual space image using indirect lighting according to an embodiment of the present invention will be described with reference to FIG. 1.

[0028] FIG. 1 is a conceptual diagram illustrating a method for playing a three-dimensional virtual space video using indirect lighting according to an embodiment of the present invention.

[0029] Referring to FIG. 1, a method for playing a three-dimensional virtual space video using indirect lighting according to an embodiment of the present invention (hereinafter, a method for playing a three-dimensional virtual space video using indirect lighting) generates a lighting effect according to the video when a user watches the video using the virtual space when playing a video (media content, etc.) in a virtual space, thereby generating a visual effect when watching and increasing the user's immersion in viewing.

[0030] Specifically, a user who wishes to use a 3D virtual space video playback service utilizing indirect lighting creates and configures various types of 3D virtual spaces, such as rooms, concert halls, and theaters. In this case, the 3D virtual space can be created in a way that allows the user to set the size, specifications, and placement of other decorations of the 3D virtual space, in addition to a method in which a pre-configured basic virtual space is provided as an option.

[0031] When a 3D virtual space is created, the user creates a video playback object to play a video in the 3D virtual space and places it in the 3D virtual space.

[0032] In addition, by placing lighting objects that generate lighting effects according to the video being played, various colors of lighting effects are generated from the lighting objects according to the video during playback, and accordingly, various effects are generated through light interference, reflection, etc., thereby providing visual effects to the user.

[0033] Hereinafter, the interaction between a virtual space video playback management server (100) and a terminal (200) according to an embodiment of the present invention will be explained with reference to FIG. 2.

[0034] FIG. 2 is an example diagram showing a communication connection between a virtual space video playback management server (100) and a terminal (200) according to an embodiment of the present invention.

[0035] Referring to FIG. 2, the virtual space video playback management server (100) and the terminal (200) are connected via a communication network. At this time, the virtual space video playback management server (100) refers to a device that performs data processing related to virtual space creation, video playback, and lighting effects in order to provide 3D virtual space video playback using indirect lighting.

[0036] The virtual space video playback management server (100) can correspond to a server that creates, provides, and manages a virtual space online, such as virtual reality, metaverse, VR, AR, MR, XR, etc.

[0037] The virtual space video playback management server (100) can create (model) various types of three-dimensional virtual spaces, such as concert halls, theaters, or rooms, in the virtual space. Additionally, the virtual space video playback management server (100) creates (models) video playback objects capable of playing videos, content, etc., within the virtual space, and lighting objects that generate lighting effects.

[0038] Additionally, the virtual space video playback management server (100) performs data processing to model the virtual space, objects for video playback, and lighting objects that generate lighting effects by performing data communication with the terminal (200).

[0039] At this time, the virtual space video playback management server (100) can be implemented not only as a single unit, but also in a form where multiple servers are linked together.

[0040] The terminal (200) can be divided into an administrator terminal that performs the placement and setting of virtual space, video playback objects, and lighting objects, and a user terminal that wants to watch video in the virtual space, and the administrator terminal and the user terminal may be the same terminal device.

[0041] The terminal (200) receives modeling data regarding a virtual space, a video playback object, and a lighting object from a virtual space video playback management server (100), and performs rendering for video playback and lighting effect generation.

[0042] The terminal (200) may refer to any type of handheld-based wireless communication device, such as a laptop, desktop, laptop, a wireless communication device with guaranteed portability and mobility, or a smartphone, tablet PC, etc., equipped with a web browser.

[0043] Additionally, the terminal (200) may include various devices for implementing virtual reality or augmented reality, such as a VR device, a motion state detection device, a camera device, an audio device, a microphone device, a display device, a body tracking device, etc.

[0044] The communication network illustrated in FIG. 2 can be implemented as a wired network such as a Local Area Network (LAN), Wide Area Network (WAN), or Value Added Network (VAN), or as any type of wireless network such as a mobile radio communication network, a satellite communication network, Bluetooth, Near Field Communication (NFC), or a 5G communication network.

[0045] Hereinafter, the structure of a virtual space video playback management server according to an embodiment of the present invention will be described with reference to FIG. 3.

[0046] FIG. 3 is a structural diagram illustrating the structure of a virtual space video playback management server (100) according to an embodiment of the present invention.

[0047] Referring to FIG. 3, the virtual space video playback management server (100) includes a communication module (110), memory (120), and a processor (140), and may further include a database (130). The communication module (110) performs information transmission and reception with a terminal (200). The communication module (110) may include a device including hardware and software necessary to transmit and receive signals, such as control signals or data signals, using a wired or wireless connection with another network device.

[0048] The memory (120) stores a virtual space video playback management program and / or instructions for performing virtual space video playback management. The name of the virtual space video playback management program is set for convenience of explanation and does not limit the function of the program by the name itself.

[0049] The memory (120) can store at least one of the information and data input to the communication module (110), the information and data required for the function performed by the processor (140), and the data generated by the execution of the processor (140).

[0050] The memory (120) should be interpreted as a general term for a non-volatile storage device that retains stored information even when power is not supplied, and a volatile storage device that requires power to retain stored information. Additionally, the memory (120) can perform the function of temporarily or permanently storing data processed by the processor (140). The memory (120) may include magnetic storage media or flash storage media in addition to a volatile storage device that requires power to retain stored information, but the scope of the present invention is not limited thereto.

[0051] The database (130) can store data related to the creation and setting of a three-dimensional virtual space, video playback objects, and lighting objects. The database (130) may constitute a part of the memory (120), but it is not necessarily located inside the virtual space video playback management server (100), and may be connected to the outside of the virtual space video playback management server (100) to perform data transmission and reception using a communication connection.

[0052] Additionally, the database (130) can store the settings of the 3D virtual space created by the user, the video playback object, and the lighting object as set values.

[0053] The processor (140) is configured to execute a virtual space video playback management program and / or instructions stored in memory (120). The processor (140) may include various types of devices for controlling and processing data.

[0054] A processor (140) may mean a data processing device embedded in hardware having a physically structured circuit to perform a function expressed by code or instructions included in a program.

[0055] In one example, the processor (140) may be implemented in the form of a microprocessor, a central processing unit (CPU), a processor core, a multiprocessor, an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA), etc., but the scope of the invention is not limited thereto.

[0056] The processor (140) is configured to execute a virtual space video playback management program or instruction to perform the following functions and procedures.

[0057] The processor (140) can create a three-dimensional modeled virtual space and create (model) and place a video playback plane object for video playback within the virtual space.

[0058] The processor (140) places a video playback plane object with a preset spacing on the wall of the virtual space and creates (models) and places a plurality of lighting objects to generate a lighting effect between the wall of the virtual space and the video playback plane object.

[0059] The processor (140) can arrange multiple lighting objects at regular intervals and create one or more aerial lighting objects in the internal space of the virtual space.

[0060] Hereinafter, the structure of a terminal according to an embodiment of the present invention will be described with reference to FIG. 4.

[0061] FIG. 4 is a structural diagram illustrating the structure of a terminal (200) according to an embodiment of the present invention.

[0062] Referring to FIG. 4, the terminal (200) includes a memory (220), an input / output module (230), and a processor (240), and may further include a communication module (210).

[0063] The communication module (210) can perform information transmission and reception with an external database or an external device. Here, the external device may be the virtual space video playback management server (100 in FIG. 2) described above.

[0064] The memory (220) stores a virtual space video playback management program. As described above, the name of the virtual space video playback management program is set for convenience of explanation and does not limit the function of the program by the name itself.

[0065] The input / output module (230) can receive information, data, etc. transmitted from the outside to the terminal (200), or output information, data, etc. possessed by the terminal (200) to the outside. For example, the input / output module (230) may include devices for delivering various multi-sensory effects to the user, such as a display, a touchpad, or a speaker.

[0066] The processor (240) executes a virtual space video playback management program stored in the memory (220). Additional descriptions of the communication module (210), memory (220), and processor (240) are replaced by the descriptions of the communication module (110 in FIG. 3), memory (120 in FIG. 3), and processor (140 in FIG. 3) described above with reference to FIG. 3.

[0067] The processor (240) executes a virtual space video playback management program and receives modeling data of a virtual space, a video playback object, and a lighting object for video playback from a virtual space video playback management server (100). Then, based on the received modeling data, it performs rendering to generate video playback and lighting effects. Additionally, the processor (240) sets the user's viewpoint at the center of the virtual space, performs texture rendering for the video to be played, and plays the video using a video playback plane object.

[0068] The processor (240) generates a lighting effect of a lighting object based on the color information of the image being played on the image playback plane object.

[0069] Additionally, the processor (240) can extract color values ​​of a plurality of border pixels included in the image at frame units or preset time intervals of the image being played on the image playback plane object, and apply the color values ​​of the border pixels to a lighting object to generate a lighting effect.

[0070] Additionally, the processor (240) divides the border of the image playback plane object into one or more zones based on the number or spacing of the lighting objects.

[0071] Then, the processor (240) matches each of one or more zones to each of a plurality of lighting objects and derives the average color value of the border pixels included in each of the plurality of zones. The processor (240) can generate a lighting effect by applying the derived average color value to the corresponding lighting object.

[0072] Additionally, the processor (240) sets the direction value of the aerial lighting object based on the video playback plane object. Then, the processor (240) sets the aerial lighting color value applied to the aerial lighting object according to the direction value of the aerial lighting object, and can generate a lighting effect by applying the aerial lighting color value to the corresponding aerial lighting object.

[0073] Additionally, the processor (240) can set one or more direction value border pixels extracted according to the direction value of the aerial lighting object among a plurality of border pixels, and apply the direction value border pixels to the aerial lighting object to generate a lighting effect.

[0074] Hereinafter, with reference to FIGS. 5 and 6, the functions of a virtual space video playback management server and a terminal for generating an indirect lighting effect when playing a video in a virtual space will be described in detail.

[0075] FIG. 5 is a conceptual diagram illustrating the functions of a virtual space video playback management server according to an embodiment of the present invention.

[0076] Referring to FIG. 5, the virtual space video playback management server (100) includes a virtual space management module (141), a video playback object management module (142), a lighting object management module (143), and a viewpoint control module (144). The virtual space management module (141) performs the function of modeling a three-dimensional virtual space and creating, setting, and managing video playback objects for playing video within the virtual space and lighting objects for creating lighting effects.

[0077] Specifically, the virtual space management module (141) provides an interface that allows the terminal (200) to set and manage the size, specifications, shape, etc. of the three-dimensional virtual space, and creates (models) a virtual space for playing an image based on the setting values ​​of the three-dimensional virtual space input using the terminal (200).

[0078] At this time, the virtual space management module (141) can create a 3D virtual space by using one or more of the following methods: a method of modeling a 3D virtual space according to a user's selected option, such as a performance hall, a theater, or a room, and a method of creating a new 3D virtual space based on a user's set value.

[0079] Accordingly, the virtual space management module (141) can create various types of three-dimensional virtual spaces, such as a performance hall, a theater, or a room, and if the user does not input separate settings, a three-dimensional virtual space in the form of a room is created by default. In addition, the virtual space management module (141) can provide a function to create and place props, decorations, etc. within the virtual space.

[0080] Additionally, the virtual space management module (141) can store information on the creation of a virtual space in the form of data with USDZ, OBJ, or FBX extensions, or create a virtual space based on data with USDZ, OBJ, or FBX extensions.

[0081] In addition, the virtual space created and configured using the virtual space management module (141) must include Diffuse color and Normal map textures. Furthermore, to use PBR (Physically Based Rendering) more clearly, additional textures such as Metallic, Specular, Ambient Occlusion, and Roughness may be included.

[0082] The video playback object management module (142) provides an interface for setting the shape, size, specifications, location, etc. of a video playback object for playing a video on a terminal (200), and creates a video playback object in a virtual space based on the setting values ​​of the video playback object input using the terminal (200).

[0083] At this time, if the virtual space is created using one of the preset options, the video playback object management module (142) can provide a preset type of video playback object as an option according to the type of virtual space.

[0084] For example, if a virtual space in the form of a performance hall or a theater is created, a video playback object in the form of a large flat screen may be provided as an option, and if a virtual space in the form of a room is created, a video playback object in the form of a television or a two-dimensional flat screen may be provided as an option.

[0085] If the user selects one of these options or directly inputs settings such as shape, size, and position, a video playback object is created in the virtual space accordingly.

[0086] In this case, the video playback object can be created to be positioned close to the wall of the virtual space, rather than in close contact with the wall of the virtual space. That is, the video playback object and the wall of the virtual space are positioned with a gap greater than a preset minimum gap and less than the maximum gap.

[0087] However, in addition to the video playback object provided as an option, the user can set various aspects such as the shape, size, specifications, and position of the video playback object.

[0088] The lighting object control module (143) provides an interface for setting the number, location, etc. of lighting objects to generate a lighting effect to the terminal (200), and creates lighting objects in a virtual space based on the input setting values.

[0089] Lighting objects can be created by distinguishing between a "basic lighting object" placed behind the video playback object, that is, between the video playback object and the virtual space wall, and an "aerial lighting object" placed in the air of the virtual space.

[0090] The default lighting object can be set to a plane or cube shape.

[0091] The number of basic lighting objects can be set from 1 to the number of border pixels of the video being played using the video playback object. At this time, the lighting object control module (143) divides the perimeter length of the video playback object based on the number of basic lighting objects and automatically places them on the back at regular intervals.

[0092] An aerial lighting object (e.g., Billboard Plane) is placed in a floating position in the air of a virtual space. The lighting object control module (143) can provide a three-dimensional lighting effect with minimal resource usage by applying Billboard technology to rotate the two-dimensional plane so that it always faces the direction of the user (camera).

[0093] The lighting object control module (143) may apply a Deferred Lighting (or Deferred Shading) technique to calculate the effect for multiple light sources. This allows for efficient rendering of multiple lights by concentrating the calculation of necessary information from the geometry pass in the post-processing stage.

[0094] The viewpoint control module (144) performs the function of setting the viewpoint of the camera (user's field of view) in virtual space.

[0095] The viewpoint control module (144) sets the user viewpoint to look at the video playback object from a central position in the created virtual space, and can be configured as a first-person and Perspective viewpoint.

[0096] The set viewpoint (camera) can rotate in various directions without restrictions on the X, Y, and Z axes. Accordingly, users can freely turn their heads in a VR environment to view ambient lighting or video.

[0097] FIG. 6 is a conceptual diagram illustrating the functions of a terminal according to an embodiment of the present invention.

[0098] Referring to FIG. 6, the terminal (200) can perform the functions of an image rendering module (241) and a lighting effect control module (242).

[0099] The terminal (200) executes a virtual space video playback management program and receives modeling data of a virtual space, a video playback object, and a lighting object for video playback from a virtual space video playback management server (100). Then, based on the received modeling data, it performs rendering to generate video playback and lighting effects.

[0100] The video rendering module (241) performs the function of rendering (mapping) the actual video to be played to a video playback object in virtual space.

[0101] The video rendering module (241) plays the video through a player provided by the OS (or a separate media player) and extracts pixel data for each frame being played.

[0102] Then, one Diffuse color texture is created and updated based on the pixel data extracted at each rendering cycle (frame by frame). At this time, the created texture can be saved at every frame or stored in a buffer.

[0103] The video rendering module (241) maps the generated texture to a video playback object, thereby constructing a screen in which the actual video is played in a three-dimensional virtual space.

[0104] In addition, the video rendering module (241) can perform real-time rendering using a GPU (graphics processing unit) by utilizing computer graphics APIs such as Metal, DirectX, and OpenGL. Through this, high-quality video can be smoothly played and displayed even in a VR environment.

[0105] The lighting effect control module (242) performs the function of extracting edge (outermost) pixel information of the image and setting a lighting color value based on the corresponding pixel color.

[0106] Specifically, the lighting effect control module (242) refers to the texture (frame-unit texture) generated by the image rendering module (241) and extracts border pixels with a certain width. During the extraction process, a Gaussian Blur or a similar blur filter is applied to mitigate excessively chaotic color differences and obtain an average color value.

[0107] The lighting effect control module (242) divides the border of the video playback object into multiple zones based on the number and placement positions of lighting objects placed in the virtual space—particularly the number and placement positions of basic lighting objects—and matches the average color value extracted from each zone to the corresponding lighting object.

[0108] Specifically, the lighting effect control module (242) divides the perimeter length of the video playback object by the number of basic lighting objects and, when basic lighting objects are placed at regular intervals on the back of the video playback object, divides the border of the video playback plane object into multiple zones based on the number or interval of basic lighting objects.

[0109] And, the lighting effect control module (242) divides the border of the video playback plane object into multiple zones corresponding to the number of basic lighting objects and matches each divided zone to a basic lighting object.

[0110] The lighting effect control module (242) derives the average color value of the border pixels included in each of the plurality of zones and applies the average color value to the basic lighting object corresponding to each of the plurality of zones to generate a lighting effect.

[0111] Additionally, the lighting effect control module (242) sets the direction value of the aerial lighting object based on the video playback object and sets the aerial lighting color value applied to the aerial lighting object according to the direction value of the aerial lighting object.

[0112] The lighting effect control module (242) extracts color data by referencing the area corresponding to the direction among the border pixels according to the direction value of the aerial lighting object (e.g., located in a specific direction such as the top left, top right, etc.). For example, the average color value of the top left area can be applied to the left aerial lighting, and the average color value of the top right area can be applied to the right aerial lighting.

[0113] Specifically, the lighting effect control module (242) sets one or more directional value border pixels extracted according to the directional value of the aerial lighting object among a plurality of border pixels, and applies the average color value of one or more directional value border pixels to the corresponding aerial lighting object to generate a lighting effect.

[0114] For example, if a first aerial lighting object is positioned at the top-left relative to a video playback object and a second aerial lighting object is positioned at the top-right relative to a video playback object, the color value of the first aerial lighting object is set by setting the color data of a pixel included in the area corresponding to the top-left position among a plurality of border pixels as the first direction value border pixel.

[0115] In addition, the color data of the pixel included in the area corresponding to the upper right position among the multiple border pixels can be set as the second direction value border pixel to set the color value of the second aerial lighting object.

[0116] In this case, using color data of pixels included in the corresponding area means either applying the average color value of multiple pixels included in the corresponding area to the aerial lighting object, or applying the color value with the highest proportion among multiple pixels included in the corresponding area to the aerial lighting object.

[0117] The lighting effect control module (242) controls each lighting object to generate a lighting effect synchronized with the video through the applied color value. Since the color value is updated whenever the video frame changes, it is possible to create a realistic effect where the ambient light changes dynamically.

[0118] Through the above series of module operations, the user can enter a virtual space (room, performance hall, etc.) and immersively view a video playing on a video playback object.

[0119] The lighting effect control module (242) applies Deferred Lighting or Deferred Shading techniques to calculate the effects for multiple light sources. This allows for the efficient rendering of multiple lights by using geometry passes to store necessary information on the screen in advance and then performing the light source calculation intensively in the post-processing stage.

[0120] The video rendering module (241) renders the actual video frame onto a 3D Plane (or screen) object, and the lighting effect control module (242) extracts and applies the video border color value, thereby implementing an Ambilight effect in which the entire space is synchronized with the video color tone.

[0121] The viewpoint control module (144) allows the camera to be moved and rotated from a first-person perspective, so that the user can freely look around the space and experience changes in lighting.

[0122] The virtual space video playback management server (100) can store and manage the settings and creation information regarding the virtual space, video playback object, and lighting object created in this way as a single data bundle, and can share this with multiple users to enable them to watch the video in the same VR environment.

[0123] In addition, the application of Billboard technology allows for the simultaneous rendering of multiple lights while reducing computational load, and the Deferred Lighting technique efficiently performs real-time calculations for various light sources.

[0124] Therefore, the present invention has the advantages of providing a highly immersive video viewing experience in a VR environment, implementing indirect lighting effects that change automatically according to the video, and enabling real-time rendering through GPU acceleration by utilizing computer graphics APIs (Metal, DirectX, OpenGL, etc.).

[0125] The functions of the above-described virtual space management module (141), video playback object management module (142), lighting object management module (143) and viewpoint control module (144) can be implemented using a virtual space video playback management server (100), and the functions of the video rendering module (241) and lighting effect control module (242) can be implemented using a terminal (200).

[0126] However, the embodiments of the present invention are not limited thereto, and the functions of each module may be implemented using a virtual space video playback management server (100) or a terminal (200), respectively, or the functions of each module may be implemented by the virtual space video playback management server (100) and the terminal (200) distributing and performing data processing processes regarding the functions of each module.

[0127] Hereinafter, a method for playing a three-dimensional virtual space image using indirect lighting according to an embodiment of the present invention will be described with reference to FIG. 7.

[0128] FIG. 7 is a flowchart of a method for playing a 3D virtual space video using indirect lighting according to an embodiment of the present invention.

[0129] Referring to FIG. 7, a method for playing a 3D virtual space video using indirect lighting includes a 3D virtual space creation step (S100), a video playback object creation step (S200), a lighting object creation step (S300), a camera viewpoint setting step (S400), a video rendering step (S500), and an indirect lighting effect creation step (S600).

[0130] In the 3D virtual space creation step (S100), the virtual space video playback management server (100) provides an interface to the terminal (200) to set and manage the size, specifications, shape, etc. of the 3D virtual space, and creates (models) a virtual space for playing a video based on the setting values ​​of the 3D virtual space input using the terminal (200).

[0131] At this time, the virtual space management module (141) can create a 3D virtual space by using one or more of the following methods: a method of modeling a 3D virtual space according to a user's selected option, such as a performance hall, a theater, or a room, and a method of creating a new 3D virtual space based on a user's set value.

[0132] Accordingly, in the 3D virtual space creation step (S100), the virtual space video playback management server (100) can create various types of 3D virtual spaces such as a performance hall, a theater, or a room, and if the user does not input separate settings, a 3D virtual space in the form of a room is created by default.

[0133] Additionally, in the 3D virtual space creation step (S100), the virtual space video playback management server (100) can provide a function to create and place props, decorations, etc. within the virtual space.

[0134] In the 3D virtual space creation step (S100), the virtual space video playback management server (100) may store creation information of the virtual space in the form of data having USDZ, OBJ, or FBX extensions, or create a virtual space based on data having USDZ, OBJ, or FBX extensions.

[0135] In the 3D virtual space generation step (S100), the virtual space generated using the virtual space image playback management server (100) must include diffuse color and normal map textures. In addition, additional textures such as metallic, specular, ambient occlusion, and roughness may be included to use PBR (Physically Based Rendering) more clearly.

[0136] In the video playback object creation step (S200), the virtual space video playback management server (100) provides an interface for setting the shape, size, specifications, location, etc. of a video playback object to play a video, and creates a video playback object in the virtual space based on the setting values ​​of the video playback object input using the terminal (200).

[0137] At this time, as described above, if the virtual space is created using one of the values ​​of the preset options, in the video playback object creation step (S200), the virtual space video playback management server (100) may provide a video playback object of a preset form as an option according to the type of virtual space.

[0138] For example, if a virtual space in the form of a performance hall or a theater is created, a video playback object in the form of a large flat screen may be provided as an option, and if a virtual space in the form of a room is created, a video playback object in the form of a television or a two-dimensional flat screen may be provided as an option.

[0139] In the video playback object creation step (S200), the virtual space video playback management server (100) creates a video playback object that does not come into contact with the wall of the virtual space and has a gap from the wall of the virtual space. That is, in the video playback object creation step (S200), the virtual space video playback management server (100) arranges the video playback object so that the video playback object and the wall of the virtual space have a gap greater than a preset minimum gap and less than a maximum gap.

[0140] In the lighting object creation step (S300), the virtual space video playback management server (100) provides an interface to the terminal (200) for setting the number and location of lighting objects to generate lighting effects, and creates lighting objects in the virtual space based on the setting values ​​of the lighting objects input using the terminal (200).

[0141] In the lighting object creation step (S300), the virtual space video playback management server (100) creates lighting objects by classifying the lighting objects into a basic lighting object placed on the back of the video playback object, that is, between the video playback object and the wall of the virtual space, and an aerial lighting object placed in the air of the virtual space.

[0142] In the lighting object creation step (S300), the virtual space video playback management server (100) creates a basic lighting object in the form of a plane or a cube. Additionally, in the lighting object creation step (S300), the virtual space video playback management server (100) can set the number of basic lighting objects from one to the number of border pixels of the video being played using the video playback object. When the number of basic lighting objects is set, the basic lighting objects are automatically placed at the rear edge of the video playback object.

[0143] In the lighting object creation step (S300), when the number of basic lighting objects is set, the virtual space video playback management server (100) divides the perimeter length of the video playback object by the number of basic lighting objects and places basic lighting objects on the back of the video playback object at regular intervals.

[0144] Additionally, in the lighting object creation step (S300), the virtual space video playback management server (100) may create one or more aerial lighting objects, and the aerial lighting objects are created to have a plane shape. Additionally, the position of the aerial lighting objects can be set by the user within the aerial area range of the virtual space.

[0145] In the camera viewpoint setting step (S400), the virtual space video playback management server (100) sets the camera viewpoint in the virtual space.

[0146] In the camera viewpoint setting step (S400), the virtual space video playback management server (100) sets the camera (viewer) viewpoint so that the video playback object is viewed from the center of the created virtual space.

[0147] The camera view set in the camera view setting step (S400) is a first-person view, and the user view can be set as a perspective view. In addition, the set user view can be rotated in various directions without restrictions on the X, Y, and Z axes.

[0148] The virtual space video playback management server (100) stores setting values ​​and creation information regarding the virtual space, video playback object, lighting object, and camera viewpoint created by the user as a single video playback virtual space data bundle. Then, the user can watch a video using the video playback virtual space they have saved, or share the saved video playback virtual space with other users so that other users can watch a video using the video playback virtual space they have created.

[0149] In the video rendering step (S500), the terminal (200) receives data regarding the modeled virtual space, video playback object, and lighting object from the virtual space video playback management server (100), and performs rendering for the video to be played in the virtual space so that the video can be played in the video playback object.

[0150] In the video rendering step (S500), the terminal (200) plays the video through a player provided by the OS (or a separate media player) and extracts pixel data for each frame being played.

[0151] In the video rendering step (S500), the terminal (200) generates and updates one diffuse color texture based on pixel data extracted at each rendering cycle (frame by frame). At this time, the generated texture may be stored at each frame or kept in a buffer.

[0152] In the video rendering step (S500), the terminal (200) maps the generated texture to a video playback object, thereby configuring a screen in which the actual video is played in a three-dimensional virtual space.

[0153] In addition, in the video rendering step (S500), the terminal (200) can perform real-time rendering using a GPU (graphics processing unit) by utilizing computer graphics APIs such as Metal, DirectX, and OpenGL. Through this, high-definition video can be smoothly played and displayed even in a VR environment.

[0154] In the indirect lighting effect generation step (S600), the terminal (200) extracts the border (outermost) pixel information of the image and sets a lighting color value based on the corresponding pixel color.

[0155] In the indirect lighting effect generation step (S600), the terminal (200) refers to a texture (frame-unit texture) generated by the image rendering module (241) and extracts border pixels with a certain width. During the extraction process, a Gaussian Blur or a similar blur filter is applied to mitigate excessively chaotic color differences and generate an average color value.

[0156] In the indirect lighting effect generation step (S600), the terminal (200) divides the border of the video playback object into multiple zones according to the number and placement positions of lighting objects placed in the virtual space—particularly the number and placement positions of basic lighting objects—and matches the average color value extracted from each zone to the corresponding lighting object.

[0157] Specifically, when the terminal (200) divides the perimeter length of the video playback object by the number of basic lighting objects and places basic lighting objects on the rear of the video playback object at regular intervals, it divides the border of the video playback plane object into multiple zones based on the number or interval of the basic lighting objects.

[0158] In the indirect lighting effect generation step (S600), the terminal (200) divides the border of the video playback plane object into multiple zones corresponding to the number of basic lighting objects and matches each divided zone to a basic lighting object.

[0159] In the indirect lighting effect generation step (S600), the terminal (200) derives the average color value of the border pixels included in each of the plurality of zones and applies the average color value to the basic lighting object corresponding to each of the plurality of zones to generate a lighting effect.

[0160] In the indirect lighting effect generation step (S600), the terminal (200) sets the direction value of the aerial lighting object based on the video playback object, and sets the aerial lighting color value applied to the aerial lighting object according to the direction value of the aerial lighting object.

[0161] In the indirect lighting effect generation step (S600), the terminal (200) extracts color data by referencing the area among the border pixels corresponding to the direction according to the direction value of the aerial lighting object (e.g., located in a specific direction such as the top left, top right, etc.). For example, the average color value of the top left area can be applied to the left aerial lighting, and the average color value of the top right area can be applied to the right aerial lighting.

[0162] Specifically, the 242 terminal (200) sets one or more directional value border pixels extracted according to the directional value of an aerial lighting object among a plurality of border pixels, and applies the average color value of one or more directional value border pixels to a corresponding aerial lighting object to generate a lighting effect.

[0163] In the indirect lighting effect generation step (S600), the terminal (200) controls each lighting object to generate a lighting effect synchronized with the video through the applied color value. Since the color value is updated whenever the video frame changes, it is possible to create a realistic effect in which the ambient light changes dynamically.

[0164] The foregoing description of the present invention is for illustrative purposes only, and those skilled in the art will understand that other specific forms can be easily modified without altering the technical spirit or essential features of the present invention. Therefore, the embodiments described above should be understood as illustrative in all respects and not restrictive. For example, each component described as a single unit may be implemented in a distributed manner, and components described as distributed may likewise be implemented in a combined form.

[0165] The scope of the present invention is defined by the claims set forth below rather than by the detailed description above, and all modifications or variations derived from the meaning and scope of the claims and equivalent concepts thereof should be interpreted as being included within the scope of the present invention.

[0166] The mode for carrying out the invention is the same as the best mode for carrying out the invention described above.

[0167] The present invention relates to a method and device for playing three-dimensional virtual space images using indirect lighting, and since it can be used in industries related to virtual reality and entertainment, it has industrial applicability.

Claims

1. As a method for performing video playback in a virtual space, Step of creating a virtual space in the form of a 3D modeled room, A step of creating a video playback object for video playback and placing the video playback object on a wall surface of the virtual space at a preset interval, A step of generating a plurality of lighting objects to generate a lighting effect between the wall surface of the virtual space and the video playback object, Step of setting the user's viewpoint at the center of the above virtual space, A step of performing texture rendering for the video to be played, and A step of generating lighting effects of the plurality of lighting objects based on color information of the video played in the above video playback object. A method for playing a 3D virtual space image using indirect lighting, including 2. In Paragraph 1, The step of generating a lighting effect of the above-mentioned lighting object is, A step of extracting color values ​​of a plurality of border pixels included in the image at frames of the image or at preset time intervals, and A step of generating a lighting effect by applying the color value of the above border pixel to the above lighting object. A method for playing a 3D virtual space image using indirect lighting, including 3. In Paragraph 2, The step of generating a lighting effect of the above-mentioned lighting object is, A step of arranging a plurality of lighting objects such that the plurality of lighting objects have a constant spacing, A step of dividing the border of the video playback object into one or more zones based on the number or spacing of the lighting objects, A step of matching each of the above one or more zones to each of the above plurality of lighting objects, A step of deriving the average color value of the border pixels included in each of the plurality of zones above, and A step of generating a lighting effect by applying the above average color value to a lighting object corresponding to each of the above plurality of zones. A method for playing a 3D virtual space image using indirect lighting, including 4. In Paragraph 3, The step of generating a lighting effect of the above-mentioned lighting object is, The step of creating one or more aerial lighting objects in the interior space of the above virtual space, A step of setting the direction value of the aerial lighting object based on the above video playback object, A step of setting an aerial lighting color value applied to the aerial lighting object according to the direction value of the aerial lighting object, A step of generating a lighting effect by applying the above aerial lighting color value to the above aerial lighting object. A method for playing a 3D virtual space image using indirect lighting, including 5. In Paragraph 4, The step of generating a lighting effect of the above-mentioned lighting object is, A step of setting one or more directional value border pixels extracted according to the directional value of the aerial lighting object among the plurality of border pixels above, A step of generating a lighting effect by applying the average color value of one or more direction value border pixels to the aerial lighting object. A method for playing a 3D virtual space image using indirect lighting, including 6. As a device for performing video playback in a virtual space, A communication module that performs information transmission and reception between a terminal and a virtual space video playback management server, Memory that stores at least one instruction, and A processor executing at least one of the above instructions Includes, The above processor creates a virtual space in the form of a 3D modeled room, creates a video playback object for video playback, arranges the video playback object at a preset interval from the wall surface of the virtual space, creates a plurality of lighting objects to generate a lighting effect between the wall surface of the virtual space and the video playback object, sets a user's viewpoint in the center of the virtual space, performs texture rendering for the video to be played, and generates a lighting effect of the plurality of lighting objects based on color information of the video played in the video playback object, a 3D virtual space video playback device utilizing indirect lighting.

7. In Paragraph 6, The above processor is, A 3D virtual space video playback device utilizing indirect lighting, which extracts color values ​​of a plurality of border pixels included in the video at frame units or preset time intervals, and applies the color values ​​of the border pixels to the lighting object to generate a lighting effect.

8. In Paragraph 7, The above processor is, A 3D virtual space video playback device utilizing indirect lighting, wherein a plurality of lighting objects are arranged such that the plurality of lighting objects have a constant interval, the border of the video playback object is divided into one or more zones based on the number or interval of the lighting objects, each of the one or more zones is matched to each of the plurality of lighting objects, an average color value of the border pixels included in each of the plurality of zones is derived, and the average color value is applied to the lighting object corresponding to each of the plurality of zones to generate a lighting effect.

9. In Paragraph 8, The above processor is, A 3D virtual space video playback device utilizing indirect lighting, which generates one or more aerial lighting objects in the interior space of the virtual space, sets a direction value of the aerial lighting objects based on the video playback object, sets an aerial lighting color value applied to the aerial lighting objects according to the direction value of the aerial lighting objects, and applies the aerial lighting color value to the corresponding aerial lighting objects to generate a lighting effect.

10. In Paragraph 9, The above processor is, A 3D virtual space image playback device utilizing indirect lighting, which sets one or more direction value border pixels extracted according to the direction value of the aerial lighting object among the plurality of border pixels, and applies the direction value border pixels to the aerial lighting object to generate a lighting effect.

Images