System for automatically generating 3D environment by adding asset

Abstract

The present specification relates to a method by which a convergence-type production device automatically generates a 3D environment, the method comprising the steps of: receiving, from a user, an input of text for generating content having the 3D environment; outputting details about the content on the basis of the text; generating a 3D background on the basis of the details about the content; and generating an asset on the basis of the details about the content.

Description

3D environment automatic generation system through asset addition

[0001] This specification relates to a system for automatically generating a 3D environment by adding assets, utilizing content automatic generation technology using visual coding.

[0002]

[0003] Due to recent technological advancements, the demand for 3D content production is surging. High-quality 3D content is essential in various fields such as games, movies, education, and the metaverse, and accordingly, numerous authoring tools and platforms are being developed to meet the diverse needs of users. However, most 3D content creation tools have several problems, including the following.

[0004] Existing 3D authoring tools place a significant burden on beginners by requiring complex workflows such as adding assets, setting attribute values, and configuring motions. Users must manually search for assets and directly adjust various attributes like position, size, angle, and shadows, which requires a considerable amount of time and effort.

[0005] Furthermore, after placing assets, users must go through several steps to add backgrounds, lighting, and interaction behaviors. In particular, the process of adding behaviors or events is very difficult to approach without development knowledge.

[0006] Furthermore, it is difficult to maintain content consistency because the standards for placed assets or lighting settings vary by user. Creating backgrounds and lighting based on a specific concept requires a high level of technical understanding.

[0007]

[0008] The purpose of the present specification is to provide a 3D environment automatic generation system that allows users to intuitively create content using visual coding.

[0009] Furthermore, the purpose of this specification is to provide a function that automatically generates backgrounds, assets, lighting, and motions through AI based on information entered by a user, verifies whether the generated content matches the user's requirements, and allows for modification.

[0010] The technical problems that this specification aims to solve are not limited to those mentioned above, and other technical problems not mentioned will be clearly understood by those skilled in the art to which this specification belongs from the detailed description of the specification below.

[0011]

[0012] One aspect of the present specification may include a method for a fused production device to automatically generate a 3D environment, comprising: receiving text from a user for generating content having the 3D environment; outputting content regarding the content based on the text; generating a 3D background based on the content; and generating an asset based on the content.

[0013] In addition, the text may include the subject of the content and the number of pages of the content.

[0014] Additionally, the step of outputting the content of the above content may include: a step of extracting the topic and the number of pages from the text; a step of inputting a prompt to an artificial intelligence model requesting the creation of scenes equal to the number of pages for the topic; and a step of outputting the content of the above content for each page based on the prompt.

[0015] In addition, the step of generating the 3D background can be generated for each page according to the content of the content output for each page.

[0016] Additionally, the step of generating the asset may include: a step of extracting information of a necessary asset based on the content of the above-mentioned content; a step of recommending the asset based on the information of the necessary asset; a step of determining whether the recommended asset has consistency with an existing placed asset; and a step of generating the asset based on the consistency.

[0017] In addition, the uniformity may be determined based on 1) the modeling quality and style of the recommended asset, 2) the size and ratio of the recommended asset, and 3) the metadata of the recommended asset.

[0018] Additionally, the method may further include the step of setting attributes and actions on the generated asset; and the step of setting lighting based on the 3D background and the generated asset.

[0019] Additionally, the above operation can be configured based on the metadata of the generated asset.

[0020] In addition, the above attributes may include transparency, shadow, and reflectance values.

[0021] Another aspect of the present specification is a convergent production device for automatically generating a 3D environment, comprising: a communication module; a memory; a display unit; and a processor for functionally controlling the communication module, the memory, and the display unit; wherein the processor receives text from a user for generating content having the 3D environment, outputs content regarding the content based on the text, generates a 3D background based on the content, and generates assets based on the content.

[0022]

[0023] According to the embodiments of the present specification, a 3D environment automatic generation system can be provided that allows a user to intuitively create content by utilizing visual coding.

[0024] In addition, according to an embodiment of the present specification, a function can be provided to automatically generate backgrounds, assets, lighting, and motions through AI based on information entered by a user, and to verify whether the generated content matches the user's requirements and to modify it.

[0025] The effects obtainable in this specification are not limited to those mentioned above, and other unmentioned effects will be clearly understood by those skilled in the art to which this specification belongs from the description below.

[0026]

[0027] FIG. 1 is a block diagram for illustrating an electronic device related to the present specification.

[0028] FIG. 2 is an embodiment to which the present specification may be applied.

[0029] FIG. 3 is an example of page generation to which the present specification can be applied.

[0030] FIG. 4 is an example of a controller (400) that can be applied to the present specification.

[0031] FIG. 5 is an example of a list of pages to which the present specification may be applied.

[0032] FIG. 6 is an example of elements to which the present specification may be applied.

[0033] FIG. 7 is an example of event detection of a fused manufacturing device to which the present specification may be applied.

[0034] FIG. 8 is an example of a method for executing a result to which the present specification can be applied.

[0035] FIG. 9 is an example of element management to which the present specification may be applied.

[0036] FIGS. 10 and FIGS. 11 are examples of element uploads to which the present specification may be applied.

[0037] FIG. 12 is a block diagram of an AI device according to one embodiment of the present specification.

[0038] FIG. 13 is an example of a recommended method pipeline to which the present specification may be applied.

[0039] FIG. 14 is a method for automatically generating 3D content according to one embodiment of the present specification.

[0040] FIG. 15 illustrates the generation of a 3D background that can be applied to the present specification.

[0041] FIG. 16 illustrates the creation of first-person content that can be applied to the present specification.

[0042] The accompanying drawings, included as part of the detailed description to aid in understanding the present specification, provide embodiments of the present specification and explain the technical features of the present specification together with the detailed description.

[0043]

[0044] Hereinafter, embodiments disclosed in this specification will be described in detail with reference to the attached drawings. Identical or similar components regardless of drawing symbols are assigned the same reference number, and redundant descriptions thereof will be omitted. The suffixes "module" and "part" used for components in the following description are assigned or used interchangeably solely for the ease of drafting the specification and do not have distinct meanings or roles in themselves. Furthermore, in describing the embodiments disclosed in this specification, if it is determined that a detailed description of related prior art could obscure the essence of the embodiments disclosed in this specification, such detailed description will be omitted. Additionally, the attached drawings are intended only to facilitate understanding of the embodiments disclosed in this specification; the technical concept disclosed in this specification is not limited by the attached drawings, and it should be understood that they include all modifications, equivalents, and substitutions that fall within the concept and technical scope of this specification.

[0045] Terms including ordinal numbers, such as first, second, etc., may be used to describe various components, but said components are not limited by said terms. These terms are used solely for the purpose of distinguishing one component from another.

[0046] When it is stated that one component is "connected" or "connected" to another component, it should be understood that while it may be directly connected or connected to that other component, there may also be other components in between. On the other hand, when it is stated that one component is "directly connected" or "directly connected" to another component, it should be understood that there are no other components in between.

[0047] A singular expression includes a plural expression unless the context clearly indicates otherwise.

[0048] In this application, terms such as “comprising” or “having” are intended to specify the existence of the features, numbers, steps, actions, components, parts, or combinations thereof described in the specification, and should be understood as not precluding the existence or addition of one or more other features, numbers, steps, actions, components, parts, or combinations thereof.

[0049] FIG. 1 is a block diagram for illustrating an electronic device related to the present specification.

[0050] The above electronic device (100) may include a wireless communication unit (110), an input unit (120), a sensing unit (140), an output unit (150), an interface unit (160), a memory (170), a control unit (180), and a power supply unit (190), etc. Since the components illustrated in FIG. 1 are not essential for implementing the electronic device, the electronic device described herein may have more or fewer components than those listed above.

[0051] More specifically, among the above components, the wireless communication unit (110) may include one or more modules that enable wireless communication between the electronic device (100) and a wireless communication system, between the electronic device (100) and another electronic device (100), or between the electronic device (100) and an external server. Additionally, the wireless communication unit (110) may include one or more modules that connect the electronic device (100) to one or more networks.

[0052] This wireless communication unit (110) may include at least one of a broadcast receiving module (111), a mobile communication module (112), a wireless internet module (113), a short-range communication module (114), and a location information module (115).

[0053] The input unit (120) may include a camera (121) or video input unit for inputting a video signal, a microphone (122) or audio input unit for inputting an audio signal, and a user input unit (123, e.g., a touch key, a mechanical key, etc.) for receiving information from a user. Voice data or image data collected from the input unit (120) may be analyzed and processed into a control command by the user.

[0054] The sensing unit (140) may include one or more sensors for sensing at least one of information within the electronic device, information about the surrounding environment surrounding the electronic device, and user information. For example, the sensing unit (140) may include at least one of a proximity sensor (141), an illumination sensor (142), a touch sensor, an acceleration sensor, a magnetic sensor, a gravity sensor (G-sensor), a gyroscope sensor, a motion sensor, an RGB sensor, an infrared sensor (IR sensor: infrared sensor), a fingerprint sensor (finger scan sensor), an ultrasonic sensor, an optical sensor (e.g., see camera (121)), a microphone (see 122), a battery gauge, an environmental sensor (e.g., a barometer, a hygrometer, a thermometer, a radiation detection sensor, a heat detection sensor, a gas detection sensor, etc.), and a chemical sensor (e.g., an electronic nose, a healthcare sensor, a biometric sensor, etc.). Meanwhile, the electronic device disclosed in this specification can utilize information sensed by at least two of these sensors in combination.

[0055] The output unit (150) is intended to generate output related to sight, hearing, or touch, and may include at least one of a display unit (151), an acoustic output unit (152), a haptic module (153), and an optical output unit (154). The display unit (151) may form a layered structure with a touch sensor or be formed integrally to implement a touch screen. Such a touch screen functions as a user input unit (123) that provides an input interface between the electronic device (100) and the user, and at the same time can provide an output interface between the electronic device (100) and the user.

[0056] The interface section (160) serves as a passage for various types of external devices connected to the electronic device (100). This interface section (160) may include at least one of a wired / wireless headset port, an external charger port, a wired / wireless data port, a memory card port, a port for connecting a device equipped with an identification module, an audio I / O (Input / Output) port, a video I / O (Input / Output) port, and an earphone port. In response to an external device being connected to the interface section (160), the electronic device (100) can perform appropriate control related to the connected external device.

[0057] Additionally, the memory (170) stores data that supports various functions of the electronic device (100). The memory (170) can store a number of application programs (or applications) running on the electronic device (100), data for the operation of the electronic device (100), and commands. At least some of these application programs may be downloaded from an external server via wireless communication. Also, at least some of these application programs may exist on the electronic device (100) from the time of shipment for the basic functions of the electronic device (100) (e.g., phone incoming and outgoing functions, message receiving and outgoing functions). Meanwhile, the application programs may be stored in the memory (170), installed on the electronic device (100), and driven by the control unit (180) to perform the operation (or function) of the electronic device.

[0058] In addition to operations related to the application program, the control unit (180) typically controls the overall operation of the electronic device (100). The control unit (180) can provide or process appropriate information or functions to the user by processing signals, data, information, etc. that are input or output through the components described above, or by running an application program stored in memory (170).

[0059] Additionally, the control unit (180) can control at least some of the components examined together with FIG. 1 in order to run an application program stored in memory (170). Furthermore, the control unit (180) can operate at least two or more of the components included in the electronic device (100) in combination with each other to run the application program.

[0060] The power supply unit (190) receives external power and internal power under the control of the control unit (180) and supplies power to each component included in the electronic device (100). This power supply unit (190) includes a battery, and the battery may be a built-in battery or a replaceable battery.

[0061] At least some of the above components may operate in cooperation with each other to implement the operation, control, or control method of an electronic device according to various embodiments described below. Additionally, the operation, control, or control method of the electronic device may be implemented on the electronic device by running at least one application program stored in the memory (170).

[0062] In the present specification, the electronic device (100) may include a terminal, a visual coding device and / or a fused manufacturing device.

[0063] FIG. 2 is an embodiment to which the present specification may be applied.

[0064] Referring to FIG. 2, the user can communicate with the convergence type production device through a terminal. For example, the terminal can be connected to the convergence type production device via WEB without a separate application, and the user can simultaneously produce 2D and 3D content through the terminal.

[0065] The fused production device receives a command to create content from a terminal via the WEB (S2010). For example, the content may include 2D and / or 3D objects.

[0066] The convergent production device creates a page for content creation (S2020). For example, the page may include a page that can be expressed in a 2D or 3D form, and the convergent production device can configure the screen by adding and placing predefined elements (Assets) or templates on each page. More specifically, events may be registered to the added elements, allowing for the addition of interactions with the content user. Through this, the user can create immersive and creative content.

[0067] FIG. 3 is an example of page generation to which the present specification can be applied.

[0068] Referring to FIG. 3, the user can receive a page display screen (300) from a convergence type production device through a terminal. For example, one content may include one or more pages. Additionally, the user can change the form of the page to 2D or 3D through a layout selection window (310) that may appear on the page display screen (300), and can add a virtual space with special functions, such as AR mode, separately according to the content user's requirements. The convergence type production device may register a separate controller (400) depending on the form of the page. Additionally, the user can change the size and aspect ratio of the page through the layout selection window (310).

[0069] Referring again to FIG. 2, the fused manufacturing device registers a controller (400) based on the page (S2030). For example, the fused manufacturing device can register a controller (400) capable of controlling and interacting with an element according to the form of the page.

[0070] FIG. 4 is an example of a controller (400) that can be applied to the present specification.

[0071] Referring to FIG. 4, when a user selects an element through a terminal, attributes corresponding to that element are displayed in the attribute window (410). The fused production device displays a registered controller (400), and the user can easily edit attributes using a mouse or touch with the controller (400). For example, the user can finely modify the attribute values ​​of the element by entering precise numerical values ​​in the attribute window (410). The user can add tabs to the attribute window (410) to perform additional connected functions. For example, additional connected functions may include the source of the element's creator or media playback information.

[0072] Referring again to FIG. 2, the fused production device places elements on the page (S2040). For example, the fused production device may place predefined elements / templates and / or additionally uploaded elements depending on the page type.

[0073] FIG. 5 is an example of a list of pages to which the present specification may be applied.

[0074] Referring to Fig. 5, one piece of content is composed of a group of multiple pages (screens), and 2D or 3D screens can be selected according to the user's requirements.

[0075] For example, a page includes (1) page attributes, (2) an event list, and (3) a resource list. More specifically, the event list contains information about events assigned to elements, and the resource list contains information about elements added to the page.

[0076] FIG. 6 is an example of elements to which the present specification may be applied.

[0077] Referring to FIG. 6, the convergence type production device can provide a user with predefined elements / templates according to the page type through a terminal. In addition, the user can upload and place additional elements.

[0078] Referring again to FIG. 2, the fused manufacturing device modifies the attribute value of the element (S2050).

[0079] For example, when a user selects an element placed on a page display screen (300), the fusion type manufacturing device displays an attribute window (410) corresponding to the element, and the user can modify the attribute value through a mouse, touch, numeric input, etc., using a controller (400) registered based on the page.

[0080] The fused manufacturing device registers an event corresponding to an element (S2060). For example, the event may include a set of "actions" and "results". More specifically, the action may define the conditions under which the event occurs. For example, the "action" may include various forms of events or calls, such as keyboard events, mouse / touch events, gesture events, area events, value events, and call events, as conditions for a function to be performed.

[0081] In addition, the result may include "functions" and "targets".

[0082] More specifically, the "function" may define attribute changes and specific actions to be performed on the "target" that is the purpose of the function when an event is activated, and may include basic attributes of elements such as position, size, rotation, and transparency, as well as control functions for media elements such as show, hide, play, stop, and pause. Additionally, it may include the user terminal's camera, GPS, accelerometer, etc., to utilize information from the external environment.

[0083] The fused manufacturing device registers actions, functions, and / or targets corresponding to the event (S2070). For example, the fused manufacturing device may register actions, functions, and / or targets based on the attribute values ​​of the elements.

[0084] The fusion type manufacturing device executes a result when an event for an element is detected based on a registered action (S2080).

[0085] FIG. 7 is an example of event detection of a fused manufacturing device to which the present specification may be applied.

[0086] Referring to FIG. 7, the fused manufacturing device can identify an event corresponding to an element, monitor the event, and detect the event. When an event is detected, the fused manufacturing device can identify attributes to execute a function corresponding to the event, and execute a result based on the function and attributes.

[0087] FIG. 8 is an example of a method for executing a result to which the present specification can be applied.

[0088] Referring to FIG. 8, the fused manufacturing device can execute "results" simultaneously, and lower "results" can be connected to upper "results" and executed continuously. Furthermore, there are no limitations on the connection of "results," and continuous functions can be performed in the next step using the execution results of the upper "results." Unlike execution methods that operate simply on a single timeline, this method of executing results can provide the user with an environment identical to actual programming techniques regarding the operation of elements, and can help in naturally learning the programming environment.

[0089] FIG. 9 is an example of element management to which the present specification may be applied.

[0090] Referring to FIG. 9, 2D and 3D elements have different attributes, making it difficult for the fused manufacturing device to control them in the same way. Therefore, the fused manufacturing device first encloses the 2D and 3D elements through an object called a "basic element," and manages the elements by extending them into "use elements" used in the authoring tool of the fused manufacturing device based on the "basic element." For example, the function of the "use element" can be configured to control and use the characteristics of the "primary element." More specifically, while FIG. 9 uses images, videos, shapes, and 3D models as examples of "use elements," any form that is advantageous for controlling and displaying the characteristics of 2D and 3D elements can be designated as a "use element."

[0091] FIGS. 10 and FIGS. 11 are examples of element uploads to which the present specification may be applied.

[0092] Referring to FIGS. 10 and 11, a fused manufacturing device (or platform) loads an uploaded element file through a loader and registers an object of the loaded element file to a "use element" that corresponds to the loader. The fused manufacturing device can place the uploaded element using the registered "use element."

[0093] Referring to FIG. 10, if the uploaded element file is an image file, the fused manufacturing device can load the image file through an image loader and register the object of the loaded image file as an image element.

[0094] Referring to FIG. 11, if the uploaded element file is a 3D model file, the fusion type manufacturing device can load the 3D model file through a 3D loader, create an object of the loaded 3D model file, create an animation, add the animation to the created object, and register the object as a 3D element.

[0095] For example, if the uploaded element file is a video file, the fused production device can create an HTMLVideoElement, add it to the screen, and then control it by registering the object to "Used Elements".

[0096] A single piece of content may include multiple pages, and each page may include a "resourceManager" that manages elements and an "eventManager" that manages action events.

[0097] Elements placed on the page have a separate depth, so the fused authoring device can adjust the depth according to the user's needs to control the display order of elements on the screen.

[0098] For example, when an element is placed in a convergent production device, it is registered in the Resource Manager of the corresponding page, allowing for management through changes to its attributes and status, as well as registration and deletion. The Resource Manager not only manages placed elements but also generates events regarding changes, enabling the invocation of connected functions based on the resource list in response to resource variations.

[0099] When an element is added, the convergent authoring device adds it to a management list to control it via a controller registered on the page, allowing the controller to recognize and manage it as a controllable element. Elements registered in the controller can have their attributes modified or be controlled using a user's mouse, gestures, touch, or external controller.

[0100] Additionally, Redo / Undo functionality may be required when users control elements. To this end, the converged authoring device can be configured to record changes to the corresponding element's state by storing controller modifications in the page's History Manager. Upon request, it can retrieve element attributes from the list of changes stored in the History Manager to update the current element attributes, thereby allowing the element's state to be restored or updated again. Since the History Manager stores changes on an element-by-element basis, memory issues may arise. Therefore, to prevent this, the converged authoring device can limit the number of items in the list of saved changes depending on the situation.

[0101] In addition, the converged production device can designate users to share content with during production, enabling them to perform tasks simultaneously. For example, when content is initially created, the converged production device can create a unique channel corresponding to the content. The converged production device designates users who can access this unique channel, and when such users access the content, they can be added to the same channel.

[0102] Users on the same channel can exchange changes in real time and synchronize authoring data in real time. To achieve this, communication can take place in real time via WebSocket or WebRTC.

[0103] However, when a new user accesses unsaved content, synchronization occurs to the state of an older version, which may result in a version discrepancy. To prevent this, when a new user is added to a shared channel and accesses the content for the first time, initial synchronization can be performed by having a specific user among the existing working users update the overall changes. After this initial synchronization is completed, the converged production device can resolve the synchronization issue of unsaved content by sharing data regarding changes in real time.

[0104] FIG. 12 is a block diagram of an AI device according to one embodiment of the present specification.

[0105] The AI ​​device (20) may include an electronic device including an AI module capable of performing AI processing, or a server including the AI ​​module. Additionally, the AI ​​device (20) may be configured to be included as at least part of the configuration of the electronic device (100) shown in FIG. 1 to perform at least part of the AI ​​processing together.

[0106] The above AI device (20) may include an AI processor (21), memory (25) and / or a communication unit (27).

[0107] The above AI device (20) is a computing device capable of learning a neural network and can be implemented as various electronic devices such as a server, desktop PC, laptop PC, tablet PC, etc.

[0108] The AI ​​processor (21) can train a neural network using a program stored in memory (25). For example, the AI ​​processor (21) can create a Recurrent Neural Networks model utilizing TensorFlow in memory (25) and train this artificial intelligence model using data that can be collected from user terminals and fused manufacturing devices.

[0109] For example, a trained artificial intelligence model may include a language model and can generate sub-models with the following tasks:

[0110] 1. Content / Template Recommendation Model: Recommends / generates customized content and templates for users by utilizing data analyzing content usage and creation information and user characteristics models.

[0111] 2. Element Recommendation Model: Recommends customized assets to content creators by utilizing user characteristic models and asset metadata.

[0112] 3. User Characteristics Analysis Model: Analyzes user characteristics through the analysis of learning interest, participation, immersion, usage, and production data.

[0113] FIG. 13 is an example of a recommended method pipeline to which the present specification may be applied.

[0114] Referring to FIG. 13, the artificial intelligence model (1300) can be trained using data collected from the platform and the fused manufacturing device.

[0115] For example, the platform can be provided to a terminal via the WEB and connected to a converged production device to provide a content creation environment to the user. Additionally, it can display content created by other users and provide a search function for this purpose. Furthermore, the platform and the converged production device can provide the user with templates that can be used in the content creation environment and provide a search function for this purpose.

[0116] The trained artificial intelligence model can generate 1) a user characteristic analysis model, 2) a content / template recommendation model, and 3) an element recommendation model. The user characteristics output by the user characteristic analysis model can be used to train the artificial intelligence model (1300).

[0117] In addition, the content / template recommendation model receives sentence input from the user, converts it into sentence data in JSON format, and can provide automatically generated content and templates to the user through a platform and a converged production device.

[0118] In addition, the element recommendation model can recommend assets to the user through a fused production device using sentence data in JSON format.

[0119] FIG. 14 is a method for automatically generating 3D content according to one embodiment of the present specification.

[0120] Referring to FIG. 14, the fused production device receives text input from a user for content creation (S1410). For example, the user may input text regarding the topic or concept for which 3D content is to be created, and how many pages of 3D content related to said topic or concept are to be created. The fused production device utilizes natural language processing to understand the user's intent and can support not only simple text but also voice or image input.

[0121] The fused production device outputs content based on text input from the user (S1420). For example, the fused production device can use an artificial intelligence model to analyze the text input by the user and derive key keywords, topics, or specific details of the content.

[0122] For example, if a user inputs text requesting the creation of 3 pages of content about history of music, the convergent production device can extract keywords related to the input text ("history of music", 3 pages) and understand the subject and structure of the content.

[0123] Referring to Table 1 below, user_prompt defines the content topic the user wants, and num_scene can specify the number of scenes (pages) to create.

[0124] context = f"""user_prompt: history of musicnum_scene: 3"""

[0125] The convergence-type production device can generate AI prompts based on text input to derive the content to be covered on each page. For example, the generated prompts can return content suitable for each page, including detailed topics of the content. Table 2 below provides examples of AI prompts.

[0126] Here is the user's prompt and the number of scenes.I want to make a 3D content that are made of multiple scenes, where they are all related with the user's prompt each containing different themes.I want you to produce three sentences in total that are related to the user_promptProduce the {1st} sentence out of three.

[0127] For example, the fused production device can request the AI ​​model to sequentially generate content corresponding to each scene (1st, 2nd, 3rd). Based on the prompt, the AI ​​model can output content suitable for the entire content. Table 3 below is an example of the generated content.

[0128] sceneSentenceList = ["The first scene transports viewers to the ancient world, where the origins of music are explored...","The second scene vividly illustrates the evolution of music through the Middle Ages...","The final scene transports viewers to the modern era, where the impact of technology on music is evident..."]

[0129] The fused production device generates a 3D background based on the content (S1430). For example, if the content requires a background, the fused production device can generate a 3D background and place it on each scene (page). More specifically, the fused production device analyzes the content and, if there is a 3D background among its existing ones that matches the concept of the content being created, places that background; if not, it can generate a new background through an artificial intelligence model and place it on the background. To this end, HDRI or Skybox technology may be used. For example, if a request is received for content with a space background, the fused production device can use HDRI to place a high-resolution image representing space as the background, and Skybox can configure the background in a form that wraps around the top, bottom, left, and right of the 3D space. Such a background determines the overall atmosphere of the content and can serve as a standard for asset placement.

[0130] FIG. 15 illustrates the generation of a 3D background that can be applied to the present specification.

[0131] Referring to FIG. 15, the fused production device can generate and place a suitable background according to the content of one page (scene).

[0132] Referring again to FIG. 14, the fused production device generates assets based on the content (S1440). First, the fused production device extracts information on necessary assets from the content. For example, if the fused production device needs to represent a "castle" in the content, it can generate a list of asset information for the castle's internal structure (tables, chairs, armor, etc.). Subsequently, the fused production device can recommend assets based on the generated list. For example, the fused production device can recommend assets based on user preferences and / or content preferences through an artificial intelligence model. More specifically, the artificial intelligence model can recommend optimal assets by analyzing themes according to the user's preferences and the content. For example, the artificial intelligence model can recommend assets that match the overall theme of the content, or it can recommend assets that reflect preferences based on the user's previous work data by analyzing user characteristics.

[0133] In addition, the convergent manufacturing device can determine, through an artificial intelligence model, whether the recommended assets are consistent with existing deployed assets. This consistency can be determined through the following three criteria.

[0134] 1. Review of modeling quality and style

[0135] AI models can evaluate the level of detail of an asset based on its polygon count. For example, a high-polygon model is suitable for backgrounds requiring realistic representation, while a low-polygon model may be suitable for a simplified style. Subsequently, the AI ​​model checks the resolution to compare the pixel density and quality of textures, thereby preventing visual discrepancies between the background and the asset. Additionally, it analyzes contour sharpness to determine if asset boundaries are distinct, and reviews color distribution and noise levels to maintain color tone consistency with the background and existing assets, ensuring the absence of unnecessary noise or defects. Through this process, the AI ​​model can be adjusted to exclude or modify assets that do not match in style.

[0136] 2. Review of Size and Proportions

[0137] The AI ​​model checks the aspect ratio of each asset to determine if proportional consistency is maintained with existing placed assets. To achieve this, it can compare the size of an asset with its ratio to the background or other assets. For example, to ensure that vehicles on a road do not appear excessively large or small, the AI ​​model can simultaneously analyze the absolute size of an asset and its relative size within the page. Assets with incorrect proportions are automatically excluded, and if necessary, their size can be adjusted while maintaining the aspect ratio. Through ratio adjustments, the AI ​​model prevents shape distortion and ensures the visual consistency of the content.

[0138] 3. Metadata-based review

[0139] AI models can utilize asset metadata to verify the match between attributes and the actual model. For example, metadata may include the asset's name, description, texture, size, and operability. For instance, an asset described in the metadata as "blue sports car on the road" can be accepted if it is actually a blue vehicle and has a size and texture suitable for a road background. The AI ​​model compares the metadata with the attributes of the actual model and can modify or exclude it if texture quality, size, color, etc., do not match. Additionally, it determines whether behavior attributes defined in the metadata (e.g., mobility) are actually implementable; if the behavior is unsuitable, it can undergo a re-recommendation process.

[0140] The review of these criteria can be performed individually or comprehensively, and subsequently, the fused production device can place consistent assets among the recommended assets.

[0141] The fused manufacturing device sets attributes and / or actions on the asset (S1450).

[0142] For example, the fused authoring device can adjust visual effects so that generated assets harmonize with or stand out against the background. More specifically, the fused authoring device can check the transparency attribute in the asset's metadata or set appropriate transparency through an AI model based on the contrast with the background. For example, the "Window" asset can be given a translucent effect by applying approximately 50% transparency.

[0143] In addition, the fused authoring device can set shadows based on the asset's position and lighting environment. More specifically, it can automatically generate shadow angles, transparency, and sizes based on the asset's position and the canvas lighting direction. For example, it can create an effect similar to a shadow on a real road beneath a sports car asset.

[0144] Additionally, the fused creator can apply metallic or glass-like reflection effects to the surface of an asset. More specifically, the fused creator can automatically calculate reflectance values ​​based on the asset's texture properties. For example, a "blue sports car" can have a high-gloss metallic reflection effect applied.

[0145] In addition, the fused authoring device can set appropriate actions for assets that require motion. The fused authoring device analyzes the asset's metadata to determine whether the asset is an operable object or requires interaction. For example, a "car" is determined to be an asset requiring motion, while a "road sign" can be considered a static asset.

[0146] More specifically, the fused creation device can determine whether an action is possible based on the name and description of the metadata. For example, it can determine that a "sports car" requires movement and that a "streetlight" is a stationary asset. Once the list of assets requiring action is organized, the fused creation device can automatically set the appropriate action based on the asset's attributes. For example, the fused creation device can set actions based on recommended action values ​​(history data or settings) specified in the metadata ("car" -> move, "light" -> blink).

[0147] The fused production device sets lighting based on the generated 3D background and / or assets (S1460). The fused production device determines whether special lighting is needed in addition to basic lighting and can provide optimized lighting effects by adjusting the properties and positions of the lighting.

[0148] For example, the fused production device can set the default lighting. The AI ​​model can set appropriate default lighting by analyzing factors such as the location of assets and the size of the background. More specifically, for outdoor scenes, it can set natural light-based sunlight lighting, while for indoor scenes, it can utilize indirect lighting to create a soft light source.

[0149] Subsequently, the fused production device determines whether special lighting is required. For example, the AI ​​model may determine that special lighting is necessary if specific assets or scenes require emphasis. More specifically, the AI ​​model can set special lighting based on key assets that need to be highlighted with a spotlight (e.g., a main character, a movie scene), or to express specific atmospheres such as horror or splendor. If special lighting is required, the AI ​​model can select and set the lighting type (e.g., point lighting, spotlight, sunlight, etc.).

[0150] The fusion type manufacturing device can 1) adjust the attribute values ​​of special lighting and 2) place special lighting.

[0151] 1. Adjust special lighting attribute values

[0152] The AI ​​model can automatically set the brightness and color of the lighting based on the theme and / or assets of the 3D background. For example, if the background has a horror theme, the special lighting can be set to a dark red light, and if the background has a summer beach theme, the special lighting can be set to a bright blue light.

[0153] Additionally, the AI ​​model can add natural lighting to match the assets and 3D backgrounds. For example, if it is summer day ocean content, special blue lighting can be added, and if it is a butcher shop scene, special red lighting can be added.

[0154] 2. Special lighting placement

[0155] The AI ​​model can position special lighting in a direction that ensures assets are visible evenly. For example, special lighting can be positioned to focus on specific assets or spaces. The AI ​​model places lights close to the center of the assets and can maximize the effect of the special lighting by utilizing spotlights or point lights.

[0156] The convergence-type production device displays completed content to the user, and the user can view and modify the content.

[0157] FIG. 16 illustrates the creation of first-person content that can be applied to the present specification.

[0158] Referring to Fig. 16, the fused production device can define the overall visual flow and user experience of the content by setting the position and movement of the camera within a 3D environment. Through this, the fused production device can automatically generate first-person content.

[0159] For example, a convergent production device can determine whether the content is in a first-person or third-person perspective by analyzing the content through an artificial intelligence model. More specifically, a first-person perspective refers to a viewpoint where the user appears to be inside the content, while a third-person perspective refers to a viewpoint where the content is observed from the outside.

[0160] For example, if the content of the fusion-type production device is about an action game, it can be judged from a first-person perspective.

[0161] The fused production device can set the position of the camera based on the determined viewpoint and the generated asset (S1440). For example, if the determined viewpoint is a first-person viewpoint and the generated asset is an asset related to a car, the camera can be positioned so that the user experiences the content from the center of the content. More specifically, the view of the driver's seat can be set as the position of the camera.

[0162] Additionally, the fused production device can set the size and angle of the camera based on the determined time point and the generated asset (S1440). For example, the fused production device can determine the scope of the content by setting the field of view (FOV) of the camera within the 3D environment. For example, the fused production device can set the angle of the camera centered on an asset or scene that the user should pay attention to.

[0163] Additionally, the fused production device can set the movement of the camera based on the determined time point and the generated asset (S1440). For example, if set as a static camera, the camera is fixed, and if set as a dynamic camera, the camera can move or rotate. Furthermore, the camera may be configured to reflect user input so that the user can move the camera using a keyboard or mouse.

[0164] Additionally, the fused production device can set physical effects on the camera based on the determined time and the generated asset (S1440). For example, it can set a shock effect where the camera shakes, or set an effect where the camera moves upward or zooms out.

[0165] Through this, the convergent production device can provide a visual experience that makes the user feel as if they have become part of the content.

[0166] The foregoing specification may be implemented as computer-readable code on a medium on which a program is recorded. A computer-readable medium includes all types of recording devices in which data that can be read by a computer system is stored. Examples of computer-readable media include Hard Disk Drives (HDDs), Solid State Disks (SSDs), Silicon Disk Drives (SDDs), ROMs, RAMs, CD-ROMs, magnetic tapes, floppy disks, optical data storage devices, etc., and also include implementations in the form of carrier waves (e.g., transmission over the Internet). Accordingly, the above detailed description should not be interpreted restrictively in all respects and should be considered exemplary. The scope of this specification should be determined by a reasonable interpretation of the appended claims, and all modifications within the equivalent scope of this specification are included within the scope of this specification.

[0167] Furthermore, although the above description has focused on the services and embodiments, this is merely illustrative and does not limit the scope of this specification. Those skilled in the art will understand that various modifications and applications not exemplified above are possible without departing from the essential characteristics of the services and embodiments. For example, each component specifically shown in the embodiments may be modified and implemented. Differences related to such modifications and applications should be interpreted as being included within the scope of this specification as defined in the appended claims.

Claims

1. In a method for a fused manufacturing device to automatically generate a 3D environment, A step of receiving text from a user to generate content having the 3D environment; A step of outputting content regarding the above content based on the above text; A step of generating a 3D background based on the content of the above-mentioned content; and A step of creating an asset based on the content of the above; A method of creation including 2. In Paragraph 1, The text above is A method of generation including the subject of the above content and the number of pages of the above content.

3. In Paragraph 2, The step of outputting the content regarding the above content A step of extracting the topic and the number of pages from the above text; A step of inputting a prompt to an artificial intelligence model requesting the creation of scenes corresponding to the number of pages regarding the above topic; and A step of outputting the content for each page based on the above prompt; A method of creation including 4. In Paragraph 3, The step of generating the above 3D background A generation method generated for each page according to the content of the above content output for each page.

5. In Paragraph 3, The step of creating the above asset is A step of extracting information on necessary assets based on the content of the above; A step of recommending the asset based on the information of the necessary asset; A step of determining whether the above-mentioned recommended asset has consistency with existing deployed assets; and Based on the above unity, a step of generating the above asset; A method of creation including 6. In Paragraph 5, The above unity A generation method determined based on 1) the modeling quality and style of the recommended asset, 2) the size and ratio of the recommended asset, and 3) the metadata of the recommended asset.

7. In Paragraph 5, Step of setting properties and behaviors on the above-mentioned generated asset; and A step of setting lighting based on the above 3D background and the above generated asset; A generation method that further includes.

8. In Paragraph 7, The above operation is A generation method configured based on the metadata of the above-mentioned generated asset.

9. In Paragraph 7, The above attributes are A generation method including transparency, shadow, and reflectance values.

10. In a fused manufacturing device for automatically generating a 3D environment, Communication module; Memory; Display unit; and A processor for functionally controlling the communication module, the memory, and the display unit; comprising, The above processor A fused production device that receives text from a user to create content having the 3D environment, outputs content regarding the content based on the text, creates a 3D background based on the content, and creates assets based on the content.

Images