3D video content production device for hologram device and method for driving the device

Abstract

Regarding a stereoscopic image content production apparatus for a hologram device and a method for driving the apparatus, the stereoscopic image content production apparatus for a hologram device of an embodiment may include a communication interface unit that communicates with a user terminal device owned by a purchaser of a hologram device that embodies hologram content, and a control unit that provides web or app services to enable the user terminal device to create, edit, or render hologram content, and that predicts demand for products and content for the hologram device based on user experience data from the use of the hologram device and hologram content, and provides custom-made products and content.

Description

[Technical Field]

[0001] The present invention relates to a stereoscopic image content producing apparatus for a hologram device and a method for operating the apparatus, and more particularly to a stereoscopic image content producing apparatus for a hologram device that provides a user experience-centric content-platform-device integrated service that enables mobile users to use holograms in the way they want, based on technologies for creating, editing, and rendering stereoscopic image content (e.g., hologram content, etc.), and a method for operating the apparatus. [Background technology]

[0002] 3D stereoscopic video can be classified by its representation method into binocular, multi-view, IP (Integral Photography), omni, panoramic, multi-view, and hologram. Hologram video is the most ideal stereoscopic vision system, capable of accurately reproducing the original 3D image in space by incorporating not only light intensity but also phase information. Therefore, the ultimate goal of 3D stereoscopic video processing technology is a fully stereoscopic hologram service, and this technology has been recognized as the ultimate crystallization of video information communication services. Currently, development of 3D broadcasting technology using digital holography is underway.

[0003] To realize this holography technology, thousands of spatial light modulators (SLMs) are required, which convert light into information signals in space, and an ultra-high-resolution display device is required to create real-time holographic images. Furthermore, an ultra-high-performance processor is also required to process the data input and output to the SLM and display device. Due to these difficulties, floating holographic images are gaining attention as an alternative technology for realizing holograms. Floating hologram technology uses a half mirror to reflect the image output from a specific display device, allowing the output image to be displayed.

[0004] However, in the past, each supplier of each element, device, content, and solution, had to request a specialized service to re-edit and create new content for each device, which caused a lot of hassle.

[0005] In addition, since currently, floating holograms require specific methods of reflection and projection on a screen or display, it is not possible to use existing digital 3D images as they are, and a process of re-editing and creating new content according to the implementation method is essential. Summary of the Invention [Problem to be solved by the invention]

[0006] An object of an embodiment of the present invention is to provide a stereoscopic video content production device for a hologram device and a method for operating the device, which provides a user experience-centric content-platform-device integrated service that helps mobile users use holograms in the way they want, based on technologies for creating, editing, and rendering stereoscopic video content (e.g., hologram content). [Means for solving the problem]

[0007] A stereoscopic image content production apparatus for a hologram device according to an embodiment of the present invention includes a communication interface unit that communicates with a user terminal device owned by a purchaser of a hologram device that embodies hologram content, and a control unit that provides a web or application service to enable the user terminal device to create, edit, or render the hologram content, and that predicts demand for products or content of the hologram device based on user experience data from the use of the hologram device or the hologram content, and provides customized products or customized content.

[0008] The control unit performs an operation for the creation, editing, or rendering when a first type of app is executed on the user terminal device, and performs a sharing operation for uploading or downloading hologram content generated by the creation, editing, or rendering when a second type of app is executed.

[0009] The control unit may collect and use, as the user experience data, data related to hologram content that is created, edited, or rendered in the user terminal device and directly provided to the hologram device.

[0010] When providing hologram content created, edited, or rendered by the user terminal device to the hologram device, the control unit can adjust pixel values ​​including color and brightness of the stereoscopic image to enhance realism based on the spatial position value where the hologram device is displayed and the implementation method of the hologram device. The control unit can adjust the pixel values ​​to represent the edges of the hologram device in black.

[0011] In addition, a method for driving a stereoscopic image content production apparatus for a hologram device according to an embodiment of the present invention includes a step in which a communication interface unit communicates with a user terminal device owned by a purchaser of a hologram device that embodies hologram content, and a step in which a control unit provides a web or application service to enable the user terminal device to create, edit, or render the hologram content, and predicts demand for products or content of the hologram device based on user experience data from the use of the hologram device or the hologram content, thereby providing customized products or customized content.

[0012] The driving method may further include a step of performing an operation for the creation, editing, or rendering when a first type of app is executed on the user terminal device, and a step of performing a sharing operation of uploading or downloading hologram content generated by the creation, editing, or rendering when a second type of app is executed on the user terminal device.

[0013] The step of providing the customized content may collect and use data related to the hologram content that is created, edited, or rendered by the user terminal device and directly provided to the hologram device as the user experience data.

[0014] The step of providing the customized content may adjust pixel values ​​including color and brightness of the stereoscopic image to enhance realism based on the spatial position value where the hologram device is displayed and the implementation method of the hologram device when providing the hologram content created, edited, or rendered by the user terminal device to the hologram device. In the adjusting of the pixel values, the control unit may adjust the pixel values ​​to represent the edges of the hologram device in black. [Effects of the Invention]

[0015] According to an embodiment of the present invention, it is possible to provide an integrated service of user experience-centered content, platform, and device that helps mobile users use holograms in the way they want, based on the technology for creating, editing, and rendering three-dimensional video content such as holograms. [Brief explanation of the drawings]

[0016] [Figure 1] 1 is a diagram illustrating a stereoscopic video content service system according to an embodiment of the present invention. [Figure 2] 2 is a diagram for simply explaining the hologram device of FIG. 1. [Figure 3] 2 is a diagram illustrating the service system of FIG. 1. [Figure 4] 2 is a diagram illustrating the service system of FIG. 1. [Figure 5] 2 is a block diagram illustrating a detailed structure of a stereoscopic image content production apparatus according to an embodiment of the present invention; [Figure 6] 4 is a flowchart illustrating a stereoscopic video content service process according to an embodiment of the present invention. [Figure 7] 4 is a flowchart illustrating a process of operating a stereoscopic image content producing apparatus according to an embodiment of the present invention. DETAILED DESCRIPTION OF THE INVENTION

[0017] Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings. FIG. 1 is a diagram showing a stereoscopic video content service system according to an embodiment of the present invention, FIG. 2 is a diagram for briefly explaining the hologram device of FIG. 1, and FIGS. 3 and 4 are diagrams showing a schematic representation of the service system of FIG. 1.

[0018] As shown in FIG. 1, a stereoscopic video content service system 90 according to an embodiment of the present invention includes some or all of a hologram device 100, a user terminal device 110, a communication network 120, a stereoscopic video content service device 130, and a third-party device 140.

[0019] Here, "including some or all" means that the stereoscopic video content service system 90 of FIG. 1 may be configured by omitting some components such as the third-party device 140, or that some or all of the components constituting the stereoscopic video content service device 130 may be integrated into the network devices (e.g., wireless switching devices) constituting the communication network 120, and in order to facilitate a full understanding of the invention, it will be described as including all.

[0020] As shown in FIG. 2, the hologram device 100 according to an embodiment of the present invention may include a device employing floating hologram technology. In other words, the hologram device 100 according to an embodiment of the present invention may be a device employing technology that reflects an image output from a specific display device using a half-mirror, i.e., a semi-transparent mirror 210, so that the output image is displayed together with a background image. More precisely, in an embodiment of the present invention, hologram content may be displayed on the hologram device 100 without including the portion that would be used as a background in a typical 2D image. Hologram images can also be used on an LG All Red transparent display, an LED FAN hologram, and a transparent mesh screen.

[0021] As shown in FIG. 2, the hologram device 100 according to the embodiment of the present invention may include a "Z"-shaped main body 200, an image panel 220 serving as a display device for outputting images from the upper side of the main body 200, and a semi-transparent mirror 210 formed at an approximately 45-degree angle from the lower side of the image panel 220. Therefore, the bottom surface, in other words, the portion of the main body 200 that forms an obtuse angle with respect to the bottom frame, may be the front when viewed in FIG. 2(b). For example, various types of real objects 230 may be placed in the portion of the bottom frame that forms an acute angle with the semi-transparent mirror 210, as shown in FIG. 2(b). The real object 230 may be a product model for advertising, an art toy for appreciation, or the like. Accordingly, when hologram content 240 is output to semi-transparent mirror 210 through image panel 220, the real object 230 and hologram content 240 may appear mixed together when viewed from one side of hologram device 100, i.e., the front, through semi-transparent mirror 210. Hologram content 240 may be implemented in various ways, such as overlapping real object 230, or in front of or behind real object 230.

[0022] Above all, in embodiments of the present invention, when the image panel 220 outputs the hologram content 240, it can output the hologram content 240 without, or excluding, the background that would normally be depicted in a typical 2D image. More precisely, the final pixel values ​​(color, brightness, etc.) of the detailed 3D image can be adjusted to enhance realism based on the position of the space in which the hologram device 100 is displayed and the implementation method of the hologram device 100. For example, it may be possible to render edges in black. In other words, customized content can be provided by recognizing the space in which the hologram is projected and the device information and adjusting (rendering) the pixel values ​​(color, brightness) of the content to be finally output. As shown in FIG. 2(b), the real object 230 and the real background of the hologram content 240 can also be implemented. The real background can be the background of the space in which the hologram device 100 is displayed, i.e., a portion of the interior. 2(a), the hologram device 100 according to the embodiment of the present invention does not have a separate mirror or glass on the rear side of the main body of the semi-transparent mirror 210, which is configured at a 45-degree angle, i.e., on the side opposite to the front side. As a result, the real object 230 and the real background of the hologram content 240 can both be realized.

[0023] Although the hologram device 100 according to an embodiment of the present invention may be configured with a fixed image panel 220, the image panel 220 may also be configured in a removable, i.e., detachable, form. Hologram images can be used with an LG all-red transparent display, an LED fan hologram, or a transparent mesh screen. Accordingly, by configuring the main body 200 in various forms, it is possible to place the user's user terminal device 110, such as a smartphone, on the top of the main body 200 with the screen facing downward, or to attach a tablet PC or the like to the top of the main body 200. For example, if a user purchases a new smartphone and has an unused old one, they can use it to create wedding photos or photos of their children as hologram content 240 according to an embodiment of the present invention and display them on the semi-transparent mirror 210. Of course, the hologram content 240 may appear vertically displayed, as if an image were displayed on a vertical panel, as shown in FIG. 2(b). Therefore, the present embodiment is not limited to any one configuration of the image panel 220.

[0024] Furthermore, the hologram device 100 according to an embodiment of the present invention can author (or create), edit, and render hologram content output from the hologram device 100 using a nearby user terminal device 110, such as a user's smartphone, and various control operations can be performed. For example, a user, or more specifically, a purchaser of the hologram device 100 according to an embodiment of the present invention, can control the operation of the hologram device 100 using his or her own user terminal device 110 and freely edit the hologram content so that it can be used in the manner desired. Of course, the user terminal device 110 can download and install an app for this purpose, for example, from the App Store or the 3D image content service device 130 of FIG. 1. In other words, the image panel 220 of the hologram device 100 can output the hologram content 240 when the hologram device 100 is in operation, with the hologram content 240 pre-stored in its internal memory. Therefore, if the content is deleted, changed, or updated in various ways due to editing in the nearby user terminal device 110, the corresponding data can be reflected.

[0025] Furthermore, when hologram device 100 outputs hologram content 240 and embodies it on semi-transparent mirror 210, or when hologram device 100 operates, it can transmit operational status data related to the embodi- ment or operation of the content to stereoscopic video content service apparatus 130 via communication network 120 of Fig. 1. Of course, data related to the user experience using hologram device 100 can also be provided to stereoscopic video content service apparatus 130 through an app on user terminal device 110 of Fig. 1 when user terminal device 110 connects, so the present invention is not limited to any one method. For example, in the latter case, hologram device 100 only needs to include a Bluetooth® communication module capable of short-range communication, which can significantly reduce manufacturing costs.

[0026] The hologram device 100 may update its internal solution, i.e., program, based on user experience data provided by the hologram device 100. For example, the hologram device 100 may update its solution based on user experience data provided by the 3D image content service apparatus 130 at the request of the 3D image content service apparatus 130. For example, if it is determined that user A uses the hologram device 100 to implement more active content, i.e., content with more screen changes, than user B, the hologram device 100 owned by user A may be updated to a solution that can implement the active content more naturally. For example, if the image panel 220 is initially set to implement 30 video frames per second, it may be updated to a solution that implements 120 video frames per second. Furthermore, if it is determined based on the user experience data that hardware replacement of the hologram device 100 is necessary, the user terminal apparatus 110 may be notified of this fact. If a simple replacement of a part is possible, the system may suggest the replacement of the part. However, if the product itself needs to be replaced, the system may recommend or recommend the product and recommend a specific product. Alternatively, the supply of a product (or model) could be adjusted based on the sales breakdown of the hologram device 100 .

[0027] The user terminal device 110 may include various types of terminal devices owned by a user who purchases and uses the hologram device 100. The user terminal device 110 may connect to the stereoscopic image content service device 130 of FIG. 1 to manage the hologram device 100 through a web service, but as described above, the hologram device 100 may also be managed through an application service. Therefore, the user terminal device 110 according to an embodiment of the present invention may include not only PC-based terminal devices such as desktop computers and laptop computers, but also mobile-based terminal devices such as smartphones, tablet PCs, and wearable devices worn on the user's wrist. For example, when a user manages the hologram device 100 through a computer, the user may connect to the web service of the stereoscopic image content service device 130 of FIG. 1 to change the hologram content and perform various operations such as creating, editing, and rendering the hologram content.

[0028] Furthermore, if the user terminal device 110 is equipped with an application capable of authoring, editing, and rendering hologram content to be implemented in the hologram device 100, the user can implement the hologram content in the hologram device 100 in any desired manner through the application. For example, as shown in FIG. 2(b), if a user wishes to change hologram content 240 to another hologram content 240, the user can execute the application (or authoring tool) and perform an operation to replace the hologram content 240 on the screen. The content can then be transmitted to the hologram device 100 and implemented through "transmit and save" so that the corresponding content can be implemented in the future. Of course, content that is changed when various operations are performed on the application screen in the user terminal device 110 may be transmitted to the hologram device 100 and implemented in the form of a "preview." Thus, by previewing content a, previewing content b, and further previewing by reflecting various edited information, the final hologram content can be implemented in the hologram device 100. The user terminal device 110 creates, edits, and renders the hologram content of the hologram device 100, and can also be seen as simultaneously performing the functions of a controller similar to a remote control. Rendering can be understood in various ways, but simply refers to adding color and effects to each surface of an object drawn on a screen in 3D computer graphics to give it a sense of depth and reality. In computer graphics, rendering refers to an "image synthesis technology" in which a computer visualizes given input values ​​(data) on a device so that a person can see them, and then outputs the result. Therefore, if the device is specialized, the rendering process can be performed separately from the content creation and editing and in conjunction with the device.

[0029] Furthermore, the user terminal device 110 may connect to the stereoscopic image content service device 130 of FIG. 1 through another app (or a second type app) separate from the app (or the first type app) for creating, editing, or rendering the hologram content 240, and upload or download hologram content generated through the first type app or video captured when the content is displayed on the hologram device 100. The former may be an authoring app, while the latter may be a sharing app. Taking into consideration the load on the communication network 120, the user terminal device 110 may share the video on a channel opened by the user to share with general users the app for creating hologram content and the video captured by the created hologram content realized through the hologram device 100, and may also provide services such as general users subscribing to the channel. Of course, the two services may be integrated into one integrated app, and the embodiments of the present invention are not limited to any one method.

[0030] The communication network 120 includes both wired and wireless communication networks. For example, the communication network 120 may be a wired or wireless Internet network or a combination thereof. Here, the wired network includes a cable network or an Internet network such as the Public Switched Telephone Network (PSTN), and the wireless communication network includes CDMA, WCDMA, GSM, Evolved Packet Core (EPC), Long Term Evolution (LTE), and a wireless broadband (Wibro) network. Of course, the communication network 120 according to an embodiment of the present invention is not limited thereto and may be used as a connection network for a next-generation mobile communication system to be realized in the future, such as a cloud computing network in a cloud computing environment or a 5G network. For example, if the communication network 120 is a wired communication network, an access point within the communication network may be connected to a telephone exchange or the like. However, if the communication network 120 is a wireless communication network, the access point may be connected to an SGSN or GGSN (Gateway GPRS Support Node) operated by a telecommunications company to process data, or to various relays such as a BTS (Base Transceiver Station), NodeB, or e-NodeB to process data.

[0031] The communication network 120 may include an access point. Access points include small base stations such as femto or pico base stations that are often installed inside buildings. Here, femto and pico base stations are classified as small base stations based on the maximum number of hologram devices 100, user terminal devices 110, etc. that they can connect to. Of course, the access point may include a short-range communication module for performing short-range communication with the hologram device 100, user terminal devices 110, etc., using ZigBee or Wi-Fi. The access point may use TCP / IP or Real-Time Streaming Protocol (RTSP) for wireless communication. Here, short-range communication may be performed using various standards, such as Wi-Fi, Bluetooth, ZigBee, infrared (IrDA), radio frequency (RF) standards such as UHF (Ultra High Frequency) and VHF (Very High Frequency), and ultra-wideband (UWB). Accordingly, the access point can extract the location of the data packet, designate the best communication path for the extracted location, and transmit the data packet along the designated communication path to the next device, for example, the stereoscopic video content service device 130. The access point can share multiple lines in a general network environment, and includes, for example, a router, a repeater, and a relay.

[0032] The stereoscopic image content service device 130 can provide an integrated platform service for holograms. The stereoscopic image content service device 130 performs service operations based on a holographic experience by gathering supply (e.g., content providers) and demand (e.g., hologram device purchasers) at once so that stereoscopic content such as holograms can be viewed immediately in a holographic format. Here, "experiencing" may refer to the user experience of the purchaser, i.e., the user, who utilizes the hologram device 100. Therefore, the stereoscopic image content service device 130 can be seen as providing an integrated service of content, platform, and device centered on user experience, which helps mobile users use holograms in the way they want, based on technologies for creating, editing, and rendering stereoscopic image content.

[0033] As shown in FIGS. 3 and 4, the stereoscopic image content service apparatus 130 or the user terminal device 110 of FIG. 1 selectively utilizes a type-specific DB 130a according to the purpose of a module (or functional block), i.e., a functional module providing a service, and manages the overall information to provide content tailored to customer needs. In other words, when a user purchases a hologram device 100, the stereoscopic image content service apparatus 130 of FIG. 1 can store or assign a serial code provided by the user terminal device 110. Accordingly, it is possible to identify which user is the (true) owner of which hologram device 100. Based on this, it is possible to monitor or collect data on which functions or operations have been used, and based on this, it is possible to optimize the operation of the hologram device 100. Here, the functional modules (or functional blocks) may be of various types, but typically may include a copyright module for copyright (or production), a usage management module for usage / management, a control module for control, etc. Of course, various other functional modules are possible, and the embodiments of the present invention are not limited to any one type.

[0034] FIG. 4 shows that the user can play and control the surrounding hologram device 100 through the user terminal device 110 of FIG. 1. The 3D video content service device 130 of FIG. 1 can operate as a cloud server. FIG. 3 shows that a serial code (membership) is used to connect and grant usage rights, providing custom-made device functions. FIG. 4 also shows that users can autonomously utilize the DB center elements to match multiple functions and service software modules according to their needs, enabling a single output (e.g., hologram 3D projection).

[0035] For example, hologram content cannot be accessed or used without a serial code provided to customers who purchase a miniaturized hologram device. The 3D image content service device 130 provides basic points when customers access the platform using the serial code, and additional points can be used for various activities or purchases. Hologram content is a service that provides differentiated, realistic 3D images rather than general 2D images, and can provide only carefully selected content through its own screening process.

[0036] With the above configuration, the embodiment of the present invention selectively utilizes a type-specific DB depending on the purpose of the module, but manages the overall information in an integrated manner, thereby improving quality through content planning tailored to demand, and enabling comprehensive service / function support for final hologram projection, thereby increasing efficiency and usability. In addition, it is possible to provide hologram services that are compatible with the current ICT environment and target mobile devices (increasing accessibility and convenience), and the existence of a ``serial code'' that qualifies users to use the integrated platform service can form a membership ecosystem, linking content, platform, and device to improve CRM effects and enable cross-marketing between affiliates (improving BM profitability compared to the existing individual quote method). Furthermore, in producing 3D image content for holograms, users can select and use the functions (modules) provided as they wish (based on 3D image content authoring / editing / rendering technology), and the 3D image content authoring / editing / rendering technology can be seen as the basis for providing a smooth integrated service that allows immediate play on the hologram device 100. Of course, in embodiments of the present invention, software technology specialized for outputting the hologram device 100 can be seen as being used. This means rendering technology, and in this case, rendering technology includes technology specialized for outputting the hologram device 100.

[0037] The third-party device 140 may include a server or computer operated by a content provider that provides solutions, i.e., programs, to the 3D image content service device 130 of FIG. 1 or hologram content for the hologram device 100. The consumer may be a hologram publicist, advertiser, or exhibitor, while the provider may be a 3D image producer. In other words, the third-party device 140 can provide content on a B2C basis or a B2B basis. Content refers to 3D images created according to the intent and characteristics of the provider (producer), such as knowledge, information, entertainment, and advertising. Examples include virtual humans, character images, animations, advertising videos, media art, and educational materials. FIG. 5 is a block diagram illustrating a detailed structure of a stereoscopic image content producing apparatus according to an embodiment of the present invention.

[0038] As shown in FIG. 5, the stereoscopic image content producing apparatus 110, 130 according to the embodiment of the present invention includes some or all of a communication interface unit 500, a control unit 510, a stereoscopic image content generating unit 520, and a storage unit 530.

[0039] Here, the term "including some or all" means that the stereoscopic video content production apparatuses 110 and 130 of Fig. 5 may be configured by omitting some components such as the storage unit 530, or some components such as the stereoscopic video content generation unit 520 may be integrated with other components such as the control unit 510, and will be described as including all components to facilitate a full understanding of the invention. Hereinafter, the stereoscopic video content production apparatuses 110 and 130 according to the embodiments of the present invention will be described assuming that they are the stereoscopic video content service apparatus 130 of Fig. 1, such as a platform server.

[0040] The communication interface unit 500 communicates with the hologram device 100, the user terminal device 110, and the third-party device 140 via the communication network 120 of Fig. 1. In the process of performing communication, the communication interface unit 500 can perform operations such as modulation / demodulation, multiplexing / demultiplexing, encoding / decoding, and scaling for converting resolution, which will be obvious to those skilled in the art and will not be described further.

[0041] The communication interface unit 500 may provide a tool or application capable of performing operations such as creating, writing, editing, and rendering hologram content of the hologram device 100 when a service request is made by the user terminal 110. Of course, the communication interface unit 500 may communicate through an application if the user terminal 110 is a mobile-based terminal, but may perform operations according to an embodiment of the present invention through a web service if the user terminal 110 is a PC-based terminal.

[0042] Furthermore, the communication interface unit 500 can communicate with the hologram device 100 and the user terminal device 110 to receive user-specific experience data of the hologram device 100 and transmit the data to the control unit 510. Here, the user-specific experience data can be considered to include all data such as what hologram content is implemented in the hologram device 100, how the user creates, edits, and renders the hologram content, and how the user plays and controls the hologram device 100.

[0043] The communication interface unit 500 may also perform an operation for updating the hologram device 100 or the user terminal device 110 under the control of the control unit 510. If an update is required for the hardware or software of the hologram device purchased by a specific user based on the user experience data, for example, an automatic update may be performed in the case of software, or a notification may be sent to the user to replace a specific component or the product itself in the case of hardware. Also, in the case of the user terminal device 110, an application or tool may be updated.

[0044] The control unit 510 performs overall control operations of the communication interface unit 500, the stereoscopic image content generation unit 520, and the storage unit 530 of Fig. 5. Typically, when the control unit 510 receives user-specific experience data of the hologram device 100 through the communication interface unit 500, the control unit 510 temporarily stores the data in the storage unit 530 and then retrieves the data to provide it to the stereoscopic image content generation unit 520 for data analysis. More precisely, the control unit 510 may classify and accumulate the user-specific experience data for each user in DB 130a of Fig. 1, and then store the data, or generate big data, and then retrieve the data to provide it to the stereoscopic image content generation unit 520 for data analysis.

[0045] The control unit 510 may also perform various operations at the request of the stereoscopic video content generator 520. For example, to meet demand for hologram content, the control unit 510 may request the third-party device 140 of FIG. 1 to create a specific type of hologram content. In other words, when a purchaser of the hologram device 100 determines that there will be an increase in video content using game characters, for example, content of a specific character, the control unit 510 may control the communication interface unit 500 of FIG. 1 to acquire content through communication with the third-party device 140 or API connection to supplement the corresponding game content. The control unit 510 may also provide content from a single creator operating in the stereoscopic video content service device 130 of FIG. 1 in the form of a subscription channel. Of course, in the case of a company, the control unit 510 may operate to provide corresponding company marketing content using the hologram device 100 displaying hologram content. Creators and companies may become purchasers of the hologram device 100, thereby improving service channels for the purchaser and simultaneously achieving promotional effects for the hologram device 100.

[0046] The stereoscopic image content generator 520 can download an app capable of authoring, editing, and rendering hologram content when a service request is made by the user terminal device 110 of Fig. 1, or can generate hologram content to be implemented in the hologram device 100 of Fig. 1 in response to an editing request from the user terminal device 110. Of course, the authoring, editing, and rendering can be provided in the form of a template, so that specific content can be selected at the request of the user terminal device 110, edited using an editing tool, and a rendering operation can be performed in which a computer converts and stores the authoring and editing data values ​​adjusted by the user into a final image. Furthermore, the hologram content can be provided to the hologram device 100 of Fig. 1 for storage and management at the user's request.

[0047] The 3D image content generator 520 also collects and analyzes user experience data generated by the hologram device 100. The 3D image content generator 520 can analyze or predict content demand through the analysis of user experience data, provide customized hologram content to users, and improve the performance of the hologram device 100. Here, "customization" can be understood as recognizing the space where the hologram is projected and the information of the hologram device 100 and adjusting (rendering) the pixel values ​​(e.g., color, brightness) of the final output content. The 3D image content generator 520 may also improve the performance of the application or tool program, i.e., the solution, provided to the user terminal device 110. For example, if a particular user frequently uses video content using game characters through the hologram device 100, the 3D image content generator 520 may update the solution to support the corresponding character content. If the performance of the hologram device 100 is inferior to the service operation the user intends to use, the 3D image content generator 520 may request hardware replacement or request notification of product replacement from the control unit 510.

[0048] Furthermore, when a user edits hologram content, the 3D image content generator 520 basically processes the background of all hologram content provided to the hologram device 100 of Fig. 1 as black. Of course, this allows editing and other operations to be performed with the background processed as black, but it is also possible to insert a block-like background image when transmitting the final edited hologram content to the hologram device 100. As mentioned above, this is to allow the real background, which is part of the exhibition space of the hologram device 100, to be seen behind when real objects and hologram content are realized in or around the real objects in the hologram device 100.

[0049] Because the hologram device 100 uses a semi-transparent mirror, the black image allows the real background to be seen. Of course, the black state can be adjusted in various ways to achieve a natural representation of the real background. In other words, in this embodiment of the present invention, the real object and hologram content of the hologram device 100 are intended to stand out by darkening the real background. However, the degree of black may also be adjusted depending on the room atmosphere, such as lighting (e.g., luminance). This can be achieved by changing the pixel value of the black pixel. Since pixel values ​​for all colors typically have 256 gray levels, a pixel value with an appropriate gray level can be selected for the black color. While automatic adjustment may be possible using an illuminance sensor, it is also possible to request the user to provide relevant information when editing hologram content and adjust the degree of black in the corresponding black image based on this information.

[0050] For example, if the exhibition space of the hologram device 100 is dark, the density of the black may be dim, but if the exhibition space is bright, the density of the black may be dark. Of course, the type of hologram content may be analyzed and the black state may be adjusted based on the analysis results to highlight the corresponding content. For example, if the hologram content is generally bright, it may be preferable to increase the density of the black image to highlight the hologram content. Conversely, if the hologram content is dark, it may be preferable to decrease the density of the black. This may change the degree to which the real object in the background is visible to the user.

[0051] In the above embodiment of the present invention, the stereoscopic image content generator 520 has been described as inserting a black background image. However, in order to generate hologram content applicable to the hologram device 100, the stereoscopic image content generator 520 according to the embodiment of the present invention may not only generate hologram content by synthesizing a black background image, but may also remove the background from content that already includes a background, such as hologram content, or adjust pixel values ​​corresponding to the background to the same color and brightness, i.e., gradation. This may vary depending on the device characteristics of the hologram device 100. In this case, instead of separately generating and synthesizing a background image, the object is simply recognized and the pixel values ​​of the background other than the object are changed, but the data processing speed can be increased by matching them. Of course, the background may be a single color, and the pixel values ​​may have the same gradation. For example, if a single color of black is generated, the pixel value may have 255 of 256 gradations. Of course, the single color may be unified with various colors other than black, and gradation values ​​may also be used in various forms depending on the type and characteristics of the hologram device 100. Therefore, in the embodiment of the present invention, it is preferable to express the background portion of the hologram content in the same color and tone.

[0052] The storage unit 530 can temporarily store various types of data processed under the control of the control unit 510. For example, the storage unit 530 can temporarily store user experience data of the hologram device 100 of Fig. 1 under the control of the control unit 510, and when editing of hologram content is performed in the user terminal device 110 of Fig. 1, the storage unit 530 can temporarily store the corresponding editing information and then provide it to the 3D image content generator 520.

[0053] In addition to the above, the communication interface unit 500, the control unit 510, the 3D image content generation unit 520, and the storage unit 530 of FIG. 5 can perform various operations, and other detailed contents have been fully explained above, so the same will be substituted here.

[0054] The communication interface unit 500, control unit 510, 3D image content generation unit 520, and storage unit 530 of FIG. 5 according to an embodiment of the present invention are configured as physically separated hardware modules, but each module may store and execute software for performing the above operations. However, since the software is a collection of software modules and each module can be implemented as hardware, the configuration is not particularly limited to software or hardware. For example, the storage unit 530 may be hardware storage or memory. However, since software-based information repository is also possible, the configuration is not particularly limited to the above.

[0055] Meanwhile, in another embodiment of the present invention, the control unit 510 may include a CPU and memory and may be formed as a single chip. The CPU may include a control circuit, an arithmetic unit (ALU), an instruction parser, and a registry, and the memory may include RAM. The control circuit performs control operations, the arithmetic unit performs binary bit information operations, and the instruction parser may include an interpreter or compiler to convert code into machine language and machine language into high-level language. The registry may be involved in storing software data. With this configuration, for example, at the initial operation of the stereoscopic video content service device 130, a program stored in the stereoscopic video content generator 520 may be copied and loaded into memory, i.e., RAM, and then executed, thereby rapidly increasing the data calculation processing speed. In the case of a deep learning model, the program may be stored in GPU memory instead of RAM and executed using the GPU to accelerate the execution speed.

[0056] FIG. 6 is a flowchart illustrating a stereoscopic video content service process according to an embodiment of the present invention. For ease of explanation, referring to FIG. 6 together with FIG. 1, a provider such as a metabus element operator or 3D creator according to an embodiment of the present invention, for example, the third-party device 140 in FIG. 1, can perform operations such as entering into partnership agreements with creators and consumers who purchase hologram devices 100 through revenue-generating activities such as content, devices, and services, for example, exhibition, public relations, and advertising market consumers (S600, S601).

[0057] Furthermore, a consumer who wishes to purchase the hologram device 100 can search for a hologram utilization method and purchase the hologram device 100, and can thereby obtain a serial code (S602 to S604).

[0058] A consumer who purchases the hologram device 100 can sign up as a member of the hologram content sharing platform, the 3D image content service device 130 of Fig. 1, through the user terminal device 110 and select a personal mobile connection, a website, or an app to use (S605, S606). For example, a PC-based terminal device can have a web connection, and a mobile-based terminal device such as a smartphone can have an app connection.

[0059] The user terminal device 110 of FIG. 1 owned by a purchaser of the hologram device 100 can connect to the stereoscopic video content service device 130 to perform various operations such as posting and purchasing (e.g., uploading and downloading) 3D content, creating 3D content, and using 3D content (S607-S609). For example, a user of the user terminal device 110 can execute an app for creating 3D content to perform operations such as creating, editing, and rendering 3D content. Subsequently, the user can share the created 3D content with the stereoscopic video content service device 130 by executing an app for sharing the created 3D content with the stereoscopic video content service device 130. Here, sharing means creating a channel, which is the user's own service space, in the stereoscopic video content service device 130 and posting various types of content related to the use of the hologram device 100 (e.g., creator content, corporate promotions, etc.) through the channel. This allows the content (e.g., a video containing a hologram) to be viewed by general users.

[0060] In addition, the stereoscopic video content service device 130 provides hologram content to the hologram device 100 at the request of the user terminal device 110, etc., and controls the play state of the hologram device 100, and accordingly, the hologram device 100 can perform a display operation using the hologram (S610, S611).

[0061] Since the manner in which each consumer, i.e., user, who purchases and uses the hologram device 100 may utilize the hologram device 100 may vary, the 3D image content service apparatus 130 may collect and database user experience data to analyze usage examples of the corresponding hologram device 100 (S612).

[0062] Subsequently, the stereoscopic image content service apparatus 130 can perform an operation for improving the UX / UI of the product or service (e.g., hologram content service, etc.) of the hologram device 100 based on the analysis result of the user experience data (S613). For example, the stereoscopic image content service apparatus 130 can increase the supply of hologram content or provide customized hologram content (e.g., content of famous characters, etc.) that is preferred by individuals based on the analysis result. Furthermore, depending on what kind of hologram device 100 a consumer has purchased and owns, the stereoscopic image content service apparatus 130 can perform an operation such as increasing the supply of hologram devices 100 or improving the performance of hardware or software resources.

[0063] In addition to the above, the hologram device 100, the user terminal device 110, the stereoscopic video content service device 130, and the third-party device 140 in FIG. 1 may perform various operations, and other details are substituted by the above.

[0064] FIG. 7 is a flowchart illustrating a process of operating the stereoscopic image content producing apparatus according to an embodiment of the present invention. For convenience of explanation, referring to FIG. 7 together with FIG. 1, a stereoscopic image content production apparatus 110, 130 of FIG. 1 according to an embodiment of the present invention, for example, a stereoscopic image content service apparatus 130, communicates with a user terminal device 110 possessed by a purchaser of a hologram device 100 that embodies hologram content (S700). Of course, the stereoscopic image content service apparatus 130 may also communicate with the user terminal device 110 to transmit hologram content created, edited, or rendered by a web service to the hologram device 100.

[0065] In addition, the 3D image content service device 130 provides web or application services that enable the user terminal device 110 to create, edit, or render hologram content, and can provide customized products and content by predicting demand for the hologram device 100's products and content based on user experience data from the use of the hologram device 100 and hologram content (S710).

[0066] The stereoscopic image content service apparatus 130 collects user experience data from the use of the hologram device 100 in which hologram content is embodied. Here, the user experience data may include data on the hologram content used in the hologram device 100, and may also include editing, authoring, and rendering information. The user experience data may also include purchase details of the hologram device 100 (e.g., product name, model information, etc.).

[0067] In addition, the stereoscopic image content service device 130 can update the performance of the hologram device 100 or the performance (e.g., application) of the user terminal device 110 that controls the hologram device 100 based on the collected user experience data.

[0068] In addition, the stereoscopic image content service apparatus 130 can write, edit, and render hologram content in the user terminal apparatus 110, and then provide the content posted using the hologram device 100 to channel subscribers such as general users when the user, i.e., a creator or company that purchased the hologram device 100, posts the content using the hologram device 100 on a channel opened by the user. The stereoscopic image content service apparatus 130 can provide content such as a video showing how the hologram content is displayed in the hologram device 100 to general users.

[0069] In addition to the above, the 3D image content producing apparatuses 110 and 130 according to the embodiments of the present invention can perform various operations, and other details have been fully explained above, so the details will be substituted here. For example, in addition to single-sided holograms, the apparatuses can also be applied to three-sided and four-sided hologram devices, and can also be used with other companies' hologram devices or transparent all-red TVs.

[0070] Meanwhile, the 3D image content production devices 110 and 130 according to the embodiments of the present invention are not limited to any particular service, as they can provide any number of additional services, such as custom-made content production services (through producer matching, auctions, subscriptions, etc.), use of the same service after separately purchasing a serial code provided at the time of product purchase, trading of NFT-type content and provision of 3D content templates, QR code (registered trademark) generation and automatic playback after shooting, NFC card recognition (for mobile) automatic playback, voice recognition control function, provision of real-time lifestyle information (time, weather, notifications), and entertainment functions (games, characters, IP-linked).

[0071] Meanwhile, even if all components constituting an embodiment of the present invention are described as being combined or operating in combination as a single unit, the present invention is not necessarily limited to such an embodiment. That is, all components may be selectively combined and operate in one or more units within the scope of the present invention. Furthermore, all components may be embodied as individual independent pieces of hardware, or some or all of the components may be selectively combined and embodied as a computer program having program modules that perform some or all of the functions combined in one or more pieces of hardware. Codes and code segments constituting the computer program can be easily construed by those skilled in the art. Such a computer program can be stored in a non-transitory computer-readable medium and read and executed by a computer to implement an embodiment of the present invention.

[0072] Here, the non-transitory readable recording medium refers to a medium that stores data semi-permanently and can be read by a device, rather than a medium that stores data for a short period of time such as a register, cache, memory, etc. Specifically, the above-mentioned program may be provided by being stored in a non-transitory readable recording medium such as a CD, DVD, hard disk, Blu-ray disc, USB, memory card, ROM, etc.

[0073] While preferred embodiments of the present invention have been illustrated and described above, the present invention is not limited to the specific embodiments described above, and it goes without saying that various modifications can be made by those skilled in the art without departing from the gist of the present invention as claimed in the claims, and such modifications should not be understood separately from the technical ideas and perspectives of the present invention.

Claims

1. a communication interface unit for communicating with a user terminal device possessed by a purchaser of a hologram device that embodies hologram content; and a control unit that provides a web or application service to enable the user terminal device to create, edit, or render the hologram content, and that predicts demand for products or content of the hologram device based on user experience data obtained by utilizing the hologram device or the hologram content to provide customized products or customized content; A stereoscopic video content production device for a hologram device.

2. The control unit performs an operation for the writing, editing, or rendering when a first type of application is executed on the user terminal device, and performs a sharing operation for uploading or downloading hologram content generated by the writing, editing, or rendering when a second type of application is executed on the user terminal device. A stereoscopic video content production apparatus for the hologram device according to claim 1.

3. The control unit collects and uses data related to hologram content that is created, edited, or rendered in the user terminal device and directly provided to the hologram device as the user experience data. A stereoscopic video content production apparatus for the hologram device according to claim 1.

4. When providing hologram content created, edited, or rendered by the user terminal device to the hologram device, the control unit adjusts pixel values ​​including color and brightness of a stereoscopic image to enhance realism based on a spatial position value where the hologram device is displayed and an implementation method of the hologram device. A stereoscopic video content production apparatus for the hologram device according to claim 1.

5. The control unit adjusts the pixel values ​​to represent the edges of the hologram device in black. A stereoscopic video content production device for the hologram device according to claim 4.

6. a communication interface unit communicating with a user terminal device possessed by a purchaser of the hologram device embodying the hologram content; and a control unit providing a web or application service to enable the user terminal device to create, edit, or render the hologram content, and predicting demand for products or content of the hologram device based on user experience data obtained by utilizing the hologram device or the hologram content to provide a customized product or content; A method for driving a stereoscopic video content production device for a hologram device, comprising:

7. performing an operation for the authoring, editing, or rendering when a first type of application is executed on the user terminal device; and performing a sharing operation of uploading or downloading the hologram content generated by the writing, editing, or rendering when the second type of application is executed on the user terminal device; A method for driving a stereoscopic video content production apparatus for the hologram device according to claim 6.

8. The step of providing the customized content includes: Data related to hologram content that is created, edited, or rendered in the user terminal device and directly provided to the hologram device is collected and used as the user experience data. A method for driving a stereoscopic video content production apparatus for the hologram device according to claim 6.

9. The step of providing the customized content includes: When the hologram content created, edited, or rendered by the user terminal device is provided to the hologram device, pixel values ​​including color and brightness of the 3D image are adjusted to enhance realism based on the spatial position value where the hologram device is displayed and the implementation method of the hologram device. A method for driving a stereoscopic video content production apparatus for the hologram device according to claim 6.

10. The step of adjusting the pixel values ​​includes: The control unit adjusts the pixel values ​​to represent the edges of the hologram device in black. A method for driving a stereoscopic video content production device for the hologram device according to claim 9.

Images