System

Abstract

To provide a system.SOLUTION: A system includes: means for acquiring a prompt sentence describing specifications or features of digital asset desired by a user; means for generating digital asset as image data or a 3D model using a generative AI model on the basis of the prompt sentence; and means for disposing the generated digital asset at a specific place in metaverse.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including a description and related instruction sentence regarding the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2022-180282 A Summary of the Invention [Problem to be solved by the invention]

[0004] While it is now possible to mass-produce digital assets, the world in which they can be used is limited, and they have not yet become widely adopted. [Means for solving the problem]

[0005] We will generate digital assets using generative AI and place the generated digital assets in the digital world (metaverse). Furthermore, by expanding the digital world (metaverse), we will contribute to the spread of the digital world where digital assets can be fully utilized. [Brief description of the drawings]

[0006] [Figure 1]1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Diagram 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. FIG. [Diagram 3] FIG. 11 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Diagram 5] FIG. 13 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 13 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 13 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 2 is a sequence diagram showing a process flow of the data processing system in the first embodiment of the first form example. [Figure 12] FIG. 11 is a sequence diagram showing a process flow of the data processing system in the application example 1 of the first embodiment. [Figure 13] FIG. 11 is a sequence diagram showing a process flow of the data processing system in the second embodiment of the second embodiment. [Figure 14] FIG. 13 is a sequence diagram showing a process flow of the data processing system in application example 2 of embodiment 2. [Figure 15] FIG. 13 is a sequence diagram showing a process flow of a data processing system in a third embodiment of the present invention. [Figure 16] FIG. 13 is a sequence diagram showing a process flow of the data processing system in the third application example of the third embodiment. [Figure 17] FIG. 13 is a sequence diagram showing the flow of processing in the data processing system in the first embodiment of the first form example when an emotion engine is combined. [Figure 18] FIG. 13 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1 of Form Example 1 when combined with an emotion engine. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0007] Hereinafter, an example of an embodiment of a system according to the technology of the present disclosure will be described with reference to the accompanying drawings.

[0008] First, the terms used in the following description will be explained.

[0009] In the following embodiments, a signed processor (hereinafter simply referred to as a "processor") may be one arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be one type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), an APU (Accelerated Processing Unit), or a TPU (TENSOR PROCESSING UNIT (registered trademark)).

[0010] In the following embodiments, a signed RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by the processor.

[0011] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0012] In the following embodiments, a communication I / F (Interface) with a code is an interface including a communication processor and an antenna. The communication I / F controls communication between multiple computers. An example of a communication standard applied to the communication I / F is a wireless communication standard including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0013] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. In addition, in this specification, the same idea as "A and / or B" is also applied when three or more things are expressed by connecting them with "and / or."

[0014] [First embodiment]

[0015] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0016] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0017] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a wide area network (WAN) and / or a local area network (LAN).

[0018] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0019] The reception device 38 includes a touch panel 38A and a microphone 38B, and receives user input. The touch panel 38A receives a touch of a pointer (for example, a pen or a finger).

[0020] The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0021] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (e.g., voice and / or text). The display 40A displays visible information such as text and images according to instructions from the processor 46. The speaker 40B outputs voice according to instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, an aperture, and a shutter, and an imaging element such as a Complementary Metal-Oxide-Semiconductor (CMOS) image sensor or a Charge Coupled Device (CCD) image sensor.

[0022] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for transmitting and receiving various types of information between the processor 46 and the processor 28 via the network 54.

[0023] FIG. 2 shows an example of main functions of the data processing device 12 and the smart device 14.

[0024] As shown in Fig. 2, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32. The specific process program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific process program 56 from the storage 32, and executes the read specific process program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific process program 56 executed on the RAM 30.

[0025] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0026] In the smart device 14, the processor 46 performs the reception output processing. The storage 50 stores a reception output program 60. The reception output program 60 is used together with the specific processing program 56 by the data processing system 10. The processor 46 reads out the reception output program 60 from the storage 50, and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0027] Next, the specific processing by the specific processing unit 290 of the data processing device 12 will be described.

[0028] "Example 1"

[0029] The present invention is a system for generating digital assets using a generative AI. Specifically, the generative AI mass-produces digital assets such as NFTs. The generated digital assets are placed in the digital world (metaverse). For example, the digital assets generated by the generative AI are placed as specific locations or objects in the metaverse.

[0030] "Example 2"

[0031] Furthermore, an embodiment of the present invention is a system for expanding the digital world (metaverse). Specifically, the digital world (metaverse) functions as a platform that can fully utilize digital assets generated by generative AI. For example, the metaverse creates and expands new regions, buildings, and objects using digital assets generated by generative AI.

[0032] The process flow of each embodiment will be described below.

[0033] "Example 1"

[0034] Step 1: Generative AI generates digital assets such as NFTs. This generation process is based on specific algorithms and rules.

[0035] Step 2: The generated digital assets are placed in the digital world (metaverse). Placement is done as a specific place or object. For example, the digital assets are placed as part of a building or scenery in the metaverse.

[0036] "Example 2"

[0037] Step 1: The digital world (metaverse) will function as a platform where digital assets generated by generative AI can be fully utilized.

[0038] Step 2: The metaverse creates and expands new regions, buildings, and objects using digital assets generated by the generative AI, for example to develop new regions and improve existing ones.

[0039] Example 1

[0040] Next, a description will be given of Example 1 of the embodiment 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal".

[0041] Conventional digital asset generation systems have a problem in that the process for users to generate specific digital assets is complicated, and it is time-consuming to place the generated digital assets in the metaverse. In addition, there is a lack of a platform for efficiently mass-producing the generated digital assets and utilizing them in the metaverse.

[0042] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0043] In this invention, the server includes a means for a user to input a prompt sentence, a means for a terminal to send a prompt sentence to the server, a means for the server to analyze the prompt sentence, a means for the server to input a prompt sentence to a generative AI model, a means for the generative AI model to generate a digital asset, a means for the server to receive the generated digital asset, a means for the server to place the digital asset in a digital world (metaverse), and a means for the user to confirm the generated digital asset in the digital world (metaverse). This allows the user to simply input a prompt sentence to generate various digital assets using the generative AI model and place them in the metaverse.

[0044] "User" means an individual or entity that uses the system to generate and verify Digital Assets within the Metaverse.

[0045] A "prompt sentence" is text that describes the specifications and characteristics of the digital asset that the user wants to generate.

[0046] A "terminal" is an electronic device that a user uses to enter prompt text and send it to a server.

[0047] A "server" is a computer system that parses prompt statements, inputs them into the generative AI model, receives the generated digital assets, and places them in the metaverse.

[0048] A “generative AI model” is an artificial intelligence model that generates digital assets based on a prompt sentence.

[0049] "Digital Asset" means an asset in digital form, such as image data or a 3D model, that is generated by a generative AI model.

[0050] The "metaverse" is a virtual space in which digital assets are located and which users can access and use.

[0051] DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0052] The present invention is a system for generating digital assets using a generative AI model and placing the digital assets in the metaverse. A specific embodiment of the system will be described below.

[0053] 1. System Configuration

[0054] The system consists of a terminal where the user inputs a prompt sentence, a server that analyzes the prompt sentence and inputs it into the generative AI model, and infrastructure for placing the generated digital assets in the metaverse.

[0055] 2. Hardware and Software Used

[0056] Terminal: A device through which a user enters a prompt, such as a computer, smartphone, or tablet.

[0057] Server: A computer system for parsing prompts and inputting them into a generative AI model. The server parses the prompts using a natural language processing (NLP) library (e.g., spaCy or NLTK).

[0058] Generative AI model: An artificial intelligence model that generates digital assets based on a prompt, such as OpenAI's GPT-4 or DALL-E.

[0059] Metaverse: A virtual space where generated digital assets are placed. Digital assets are placed using the metaverse API.

[0060] 3. Data processing and calculation

[0061] Prompt analysis: The server analyzes the prompt entered by the user using natural language processing technology. For example, the server extracts the keyword "blue dragon" from the prompt "Please generate an NFT of a blue dragon."

[0062] Digital asset generation: The generative AI model generates digital assets based on the parsed prompt sentences. The generated digital assets are output as image data or 3D models.

[0063] Placing digital assets: The server places the generated digital assets in a specific location in the metaverse. Using the metaverse API, the server places the digital assets in the virtual space.

[0064] 4. Specific Examples

[0065] The user inputs a prompt sentence saying "I want you to generate a red phoenix NFT." The terminal sends this prompt sentence to the server. The server parses the prompt sentence and extracts the keyword "red phoenix." The server then inputs this keyword into the generative AI model and generates a red phoenix digital asset. The generated digital asset is placed in a specific area in the metaverse by the server. The user can access the metaverse and check the generated red phoenix NFT.

[0066] In this way, users can simply enter a prompt to have a generative AI model generate various digital assets and place them in the metaverse.

[0067] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0068] Step 1:

[0069] The user enters a prompt sentence.

[0070] Specifically, the user enters a prompt such as "Please generate an NFT of a blue dragon" into the device's input form.

[0071] Input: Prompt text (e.g. "I want you to generate an NFT of a blue dragon")

[0072] Output: The prompt text entered in the terminal

[0073] Step 2:

[0074] The terminal sends a prompt to the server.

[0075] In concrete terms, when the user clicks the send button, the terminal sends a prompt text to the server using an HTTP request.

[0076] Input: The prompt text entered into the terminal

[0077] Output: The prompt text sent to the server.

[0078] Step 3:

[0079] The server parses the prompt.

[0080] Specifically, the server uses a natural language processing (NLP) library (e.g., spaCy or NLTK) to parse the prompt and extract the keyword "blue dragon."

[0081] Input: The prompt text sent to the server

[0082] Output: Parsed keywords (e.g. "blue dragon")

[0083] Step 4:

[0084] The server inputs a prompt sentence into the generative AI model.

[0085] Specifically, the server sends an API request to input the analyzed keywords into a generative AI model (e.g., GPT-4 or DALL-E).

[0086] Input: Parsed keyword

[0087] Output: The request sent to the generative AI model

[0088] Step 5:

[0089] A generative AI model generates digital assets.

[0090] Specifically, the generative AI model runs an image generation algorithm internally to generate the "blue dragon" digital asset.

[0091] Input: The request sent to the generative AI model

[0092] Output: Generated digital assets (e.g. image data of a blue dragon)

[0093] Step 6:

[0094] The server receives the generated digital asset.

[0095] Specifically, the server receives the digital assets returned from the generative AI model and stores them in a database.

[0096] Input: Generated digital assets

[0097] Output: Digital assets stored in a database

[0098] Step 7:

[0099] The server places the digital assets in the metaverse.

[0100] Specifically, the server uses the metaverse's API to place the generated digital assets in specific locations in the virtual space.

[0101] Input: Digital assets stored in a database

[0102] Output: Digital assets placed in the metaverse

[0103] Step 8:

[0104] The user views the generated digital assets in the metaverse.

[0105] Specifically, users will access the metaverse using a VR headset or PC and view the placed blue dragon NFT.

[0106] Input: Digital assets placed within the metaverse

[0107] Output: Digital assets viewed by the user

[0108] (Application example 1)

[0109] Next, a description will be given of Application Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0110] In conventional digital asset generation systems, it was difficult for users to easily generate and place digital assets suitable for specific locations in a virtual store. In addition, there was a lack of means to effectively utilize the generated digital assets in the metaverse. This resulted in a problem that virtual store owners could not quickly create and place their own digital assets, making store operations in the metaverse inefficient.

[0111] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0112] In this invention, the server includes a means for generating digital assets using generative AI, a means for arranging the generated digital assets in a digital world (metaverse), a means for expanding the digital world (metaverse), a means for generating digital assets based on a prompt sentence input by a user, and a means for arranging the generated digital assets in a specific location in a virtual store. This makes it possible for a user to generate and quickly arrange suitable digital assets in a virtual store simply by inputting a prompt sentence.

[0113] "Generative AI" is an artificial intelligence system that automatically generates digital assets based on prompts entered by users.

[0114] A "digital asset" is any digital form of property used within the digital world (metaverse), including, for example, NFTs.

[0115] The "digital world (metaverse)" is a virtual space built on the Internet, and is a platform where users can utilize digital assets to take action.

[0116] A "prompt" is text that a user inputs into a generative AI that dictates the content of the digital asset to be generated.

[0117] A "virtual store" is a virtual store that exists in the digital world (metaverse) and is a place where users can purchase goods and services.

[0118] A "means of placement" is a technical means for placing a generated digital asset at a specific location within the digital world (metaverse).

[0119] The following system configuration will be described as an embodiment of the present invention.

[0120] First, the server generates a digital asset using a generative AI. A generative AI is an artificial intelligence system that automatically generates a digital asset based on a prompt sentence entered by a user. A user enters a prompt sentence using a device such as a smartphone, smart glasses, or a head-mounted display. For example, by entering the prompt sentence "a luxurious virtual store entrance," the generative AI generates a digital asset based on that instruction.

[0121] The generated digital assets are placed in the digital world (metaverse) by the server. The digital world (metaverse) is a virtual space built on the Internet and is a platform where users can use digital assets to take part in activities. The server provides the technical means to place the generated digital assets in specific locations within the virtual store.

[0122] Specifically, the server does the following:

[0123] 1. Receive the prompt text entered by the user.

[0124] 2. Use generative AI to generate digital assets based on the prompt.

[0125] 3. Place the resulting digital asset in a designated location within the metaverse.

[0126] The implementation of this system uses the following hardware and software:

[0127] Hardware: Smartphones, smart glasses, head-mounted displays

[0128] Software: OpenAI API, Metaverse API, Python

[0129] As a specific example, when a user inputs the prompt sentence "A luxurious virtual store entrance," the generative AI generates a luxurious digital asset based on that instruction. The generated digital asset is placed at the entrance of the virtual store by the server. This allows the user to generate and quickly place a suitable digital asset in a virtual store simply by inputting a prompt sentence.

[0130] The flow of the specific process in the application example 1 will be described with reference to FIG.

[0131] Step 1:

[0132] The user uses the terminal to input a prompt sentence, which includes information indicating the content and location of the digital asset desired by the user.

[0133] Specifically, the user inputs a prompt sentence indicating the content of the digital asset to be generated using a terminal such as a smartphone, smart glasses, or a head-mounted display. For example, the user inputs the prompt sentence "a luxurious virtual store entrance."

[0134] Input: Prompt text (e.g. "A luxurious virtual store entrance")

[0135] Output: The prompt is sent to the server.

[0136] Step 2:

[0137] The server receives the prompt text and sends it to the generative AI.

[0138] Specifically, the server receives a prompt sentence sent by a user and sends the prompt sentence to the generative AI, which then starts a process to generate a digital asset based on the prompt sentence.

[0139] Input: prompt statement

[0140] Output: The prompt is sent to the generative AI.

[0141] Step 3:

[0142] A generative AI generates digital assets based on the prompt text.

[0143] Specifically, the generative AI analyzes the prompt text and generates an appropriate digital asset based on its content. For example, based on the prompt text "A high-end virtual store entrance," it generates a high-end digital asset.

[0144] Input: prompt statement

[0145] Output: Generated digital assets

[0146] Step 4:

[0147] The server receives the generated digital assets and places them in specific locations within the digital world (metaverse) using the metaverse API.

[0148] Specifically, the server receives the digital assets generated by the generative AI and uses the metaverse API to place the digital assets in a specified location in the virtual store. For example, a luxurious digital asset is placed at the entrance of the virtual store.

[0149] Input: The generated digital asset, and the location specified in the prompt.

[0150] Output: A digital asset is placed in a specific location in the metaverse.

[0151] Step 5:

[0152] A user uses a terminal to check a virtual store in the metaverse.

[0153] Specifically, users use devices such as smartphones, smart glasses, and head-mounted displays to check the virtual store in the metaverse and verify that the generated digital assets are correctly placed.

[0154] Input: Virtual store information in the metaverse

[0155] Output: User confirms placement of digital assets

[0156] Example 2

[0157] Next, a second embodiment of the second embodiment will be described. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0158] In the conventional digital world (metaverse), the means by which users could generate and place new digital assets were limited, making it difficult to expand the metaverse. In addition, there was a lack of efficient methods for managing and placing the generated digital assets. This made it difficult for users to maximize their creativity, and restricted the development of the metaverse.

[0159] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0160] In this invention, the server includes a means for a user to input a prompt sentence, a means for the server to receive the prompt sentence and input it to the generative AI model, a means for the server to generate a digital asset using the generative AI model, a means for the server to store the generated digital asset in a database, a means for a terminal to obtain the digital asset from the server, a means for a user to place the digital asset in a digital world (metaverse) using the terminal, and a means for expanding the digital world (metaverse). This allows a user to efficiently generate digital assets using the generative AI model and freely place them in the metaverse.

[0161] A "User" is an individual or entity that utilizes the system to enter prompt statements and place generated digital assets within the metaverse.

[0162] A "prompt sentence" is a textual instruction sentence that a user inputs to instruct a generative AI model to generate a digital asset.

[0163] "Server" means a computer system for receiving prompt sentences, generating digital assets using a generative AI model, and storing them in a database.

[0164] A "generative AI model" is an artificial intelligence model for generating digital assets based on prompt sentences entered by a user.

[0165] "Digital assets" are digital content such as 3D models, textures, and animations that are generated by generative AI models and used within the metaverse.

[0166] "Database" means an information management system for storing generated digital assets and their metadata.

[0167] A "terminal" is hardware such as a computer or VR device that a user uses to obtain digital assets from the server and place them in the metaverse.

[0168] The "metaverse" is a virtual space where generated digital assets can be placed and where users can interact with each other.

[0169] A "means for placement" is a method or tool that allows a user to place a digital asset at a specific location in the metaverse using a terminal.

[0170] An "expansion tool" is a method or tool for increasing areas and objects within the metaverse and expanding the virtual space using generated digital assets.

[0171] The present invention is a system for generating digital assets using a generative AI model and placing them in a digital world (metaverse). A specific embodiment of the system will be described below.

[0172] First, the user inputs a prompt on the Metaverse interface. The prompt instructs the generative AI model on the content of the digital asset they want to generate. For example, they input a prompt such as "I want you to generate a 3D model of a futuristic city."

[0173] Next, the server receives the prompt sentence sent by the user. The received prompt sentence is input to the generative AI model. For example, a general artificial intelligence model (such as GPT-4 or DALL-E) can be used as the generative AI model.

[0174] The server uses a generative AI model to generate digital assets based on the prompt text. The generated digital assets can range from 3D models, textures, animations, and more. The generated digital assets are stored in a database on the server. The database also stores metadata for the generated digital assets (such as the generation date and time, the prompt text, etc.).

[0175] After that, the terminal obtains the digital asset from the server in response to the user's request. The obtained digital asset is temporarily stored in the terminal's local storage. The terminal can be the user's PC or VR device.

[0176] Users place digital assets acquired using their devices in the metaverse. For example, a 3D model of a futuristic city can be placed as a new area. In this case, users can intuitively operate the assets using a VR device or PC.

[0177] For example, if a user inputs the prompt "I want you to generate a 3D model of a futuristic city," the server uses the generative AI model to generate a 3D model of a futuristic city. This 3D model is stored on the server and sent to the user's device. The user can place this 3D model in the metaverse and use it as a new region.

[0178] Examples of prompts include the following:

[0179] "I want you to generate a 3D model of a medieval castle."

[0180] "I want you to generate a 3D model of a futuristic city."

[0181] "I want you to generate a fantasy forest texture."

[0182] This system allows users to efficiently generate digital assets using generative AI models and place them freely in the metaverse, making it easier to expand the metaverse and maximizing users' creativity.

[0183] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0184] Step 1:

[0185] The user enters a prompt sentence.

[0186] A user enters "I want you to generate a 3D model of a futuristic city" into a text box on the Metaverse interface and clicks the send button. The entered prompt is sent to the server.

[0187] Step 2:

[0188] The server receives the prompt sentence and inputs it into the generative AI model.

[0189] The server receives the prompt text sent by the user. It sends the received prompt text as a request to the API of the generative AI model. The input is the prompt text, and the output is the digital asset data generated by the generative AI model.

[0190] Step 3:

[0191] The server generates the digital assets using the generative AI model.

[0192] The server receives the digital asset data returned by the generative AI model and converts it into a 3D model file format (e.g., OBJ or FBX). The input is the data from the generative AI model, and the output is the converted 3D model file.

[0193] Step 4:

[0194] The server stores the generated digital assets in a database.

[0195] The server saves the generated 3D model to a database and logs the saving completion status. The input is the 3D model file and the output is the saving completion status to the database.

[0196] Step 5:

[0197] The terminal obtains the digital asset from the server.

[0198] The terminal sends a request to the server in response to the user's operation and downloads the generated 3D model. After the download is complete, it is saved in local storage. The input is the request to the server, and the output is the 3D model file saved in local storage.

[0199] Step 6:

[0200] A user places a digital asset in the metaverse using a terminal.

[0201] The user puts on the VR device and switches to edit mode in the metaverse. The user selects the acquired 3D model and places it in the desired position by dragging and dropping it. After placing it, the user clicks the save button to confirm the changes. The input is the 3D model file stored in the local storage, and the output is the 3D model placed in the metaverse.

[0202] (Application example 2)

[0203] Next, a description will be given of Application Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0204] In conventional metaverse platforms, the process for users to generate and deploy digital assets is complicated and requires specialized knowledge, making it difficult for general users to use. In addition, in virtual stores, it is difficult to freely customize store layouts and product displays, preventing users from fully expressing their creativity.

[0205] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0206] In this invention, the server includes a means for generating digital assets using a generative AI, a means for placing the generated digital assets in a digital world (metaverse), a means for expanding the digital world (metaverse), a means for the generative AI to generate digital assets in response to a user inputting a prompt sentence, and a means for placing the generated digital assets in a virtual store. This allows a user to generate digital assets by simply inputting a prompt sentence without requiring specialized knowledge, and to freely place them in a virtual store.

[0207] "Generative AI" is artificial intelligence that automatically generates digital assets based on prompts entered by users.

[0208] "Digital assets" is a general term for virtual objects and resources used within the digital world (metaverse).

[0209] The "digital world (metaverse)" is a virtual space built on the Internet, and is a platform where users can interact and engage in activities through avatars.

[0210] A "prompt sentence" is text that a user enters to instruct the generative AI on the content of the digital asset they want to generate.

[0211] A "virtual store" is a virtual store that exists in the digital world (metaverse) where users can browse and purchase products.

[0212] A "means of placement" is a method or technology for placing a generated digital asset at a specific location within the digital world (metaverse).

[0213] "Augmentation tools" are methods and technologies for expanding the scope and content of the digital world (the metaverse).

[0214] The system for implementing this invention includes a means for generating digital assets using a generative AI, placing them in a digital world (metaverse), and further expanding the digital world. Specifically, the generative AI generates digital assets when a user inputs a prompt sentence, and the generated digital assets can be placed in a virtual store.

[0215] 1. System Programming

[0216] The system uses the following hardware and software:

[0217] Hardware: Smartphones, smart glasses, head-mounted displays

[0218] Software: OpenAI API, Unreal Engine

[0219] 2. Explanation of program processing

[0220] The server receives the prompt text entered by the user and sends it to the generative AI. The generative AI generates a digital asset based on the prompt text and returns the data to the server. The server reads the returned digital asset data using Unreal Engine and places it in the specified location.

[0221] The specific process is as follows:

[0222] 1. The user uses a smartphone or head-mounted display to input a prompt describing the digital asset they want to generate.

[0223] 2. The server sends this prompt to the OpenAI API, and the generative AI generates the digital asset data.

[0224] 3. The generated digital asset data is returned to the server, which uses the Unreal Engine to load it and place it in the specified location within the virtual store.

[0225] 3. Specific Examples

[0226] For example, if the user types "I want to create a new shelf", the prompt text might look like this:

[0227] Example prompt:

[0228] "Generate a new product shelf. The shelf is made of wood and has 5 shelves, with different types of products on each shelf."

[0229] When this prompt is sent to the generative AI, the AI ​​generates a five-tier wooden shelf with different types of products on each shelf, and returns a digital asset that the server can then place in a virtual store.

[0230] In this way, users can generate digital assets by simply entering prompt sentences without the need for specialized knowledge, and place them freely in a virtual store.

[0231] The flow of the specific process in the application example 2 will be described with reference to FIG.

[0232] Step 1:

[0233] The user uses a smartphone or a head-mounted display to input a description of the digital asset to be generated as a prompt sentence.

[0234] Input: A prompt to be entered by the user (e.g. "Create a new shelf. The shelf is made of wood and has five shelves, each with a different type of product on it.")

[0235] Output: The prompt text is sent to the server.

[0236] Step 2:

[0237] The server sends the received prompt text to the OpenAI API, and the generative AI generates data for the digital asset.

[0238] Input: The prompt received by the server

[0239] Output: Digital asset data generated by generative AI

[0240] Specific operation: The server sends the prompt text to the OpenAI API, and the generative AI generates a digital asset based on the prompt text.

[0241] Step 3:

[0242] The generated digital asset data is returned to the server.

[0243] Input: Digital asset data generated by generative AI

[0244] Output: The digital asset data is returned to the server.

[0245] Specific operation: The generative AI returns the generated digital asset data to the server.

[0246] Step 4:

[0247] The server loads the returned digital asset data using the Unreal Engine and places it in the specified location within the virtual store.

[0248] Input: The digital asset's data returned by the server

[0249] Output: Digital assets are placed in a virtual store.

[0250] Specific operation: The server uses Unreal Engine to load the digital asset data and place it in the specified location in the virtual store.

[0251] Step 5:

[0252] Users can view the digital assets placed in the virtual store through their smartphones or head-mounted displays.

[0253] Input: Digital assets placed in a virtual store

[0254] Output: A representation of the digital asset for the user to see

[0255] Specific operation: The user uses a smartphone or a head-mounted display to visually check the digital assets placed in the virtual store.

[0256] Furthermore, an emotion engine that estimates the emotion of the user may be combined. That is, the identification processing unit 290 may estimate the emotion of the user using the emotion identification model 59, and perform identification processing using the emotion of the user.

[0257] "Example 1"

[0258] In one embodiment of the present invention, a generative AI and an emotion engine are combined. The generative AI generates digital assets. Meanwhile, the emotion engine recognizes the user's emotions. Specifically, the emotion is estimated from the user's facial expressions, tone of voice, text input, and the like. The output of this emotion engine is fed back to the generative AI, and the generation of the digital asset is adjusted according to the user's emotions. For example, when the user feels joy, the generative AI generates a digital asset with a bright color tone. Conversely, when the user feels sad, the generative AI generates a digital asset with a dark color tone.

[0259] "Example 2"

[0260] In another embodiment of the present invention, a generative AI, an emotion engine, and a digital world (metaverse) are combined. The generative AI generates digital assets, and the emotion engine recognizes the user's emotions. The digital world (metaverse) then arranges the generated digital assets and adjusts the arrangement according to the user's emotions. For example, when a user feels joy, the digital world (metaverse) arranges digital assets in bright tones around the user. Conversely, when a user feels sad, the digital world (metaverse) arranges digital assets in dark tones around the user.

[0261] The process flow of each embodiment will be described below.

[0262] "Example 1"

[0263] Step 1: The emotion engine recognizes the user's emotions. Specifically, it estimates emotions from the user's facial expressions, tone of voice, text input, etc.

[0264] Step 2: The output of the emotion engine is fed back to the generative AI.

[0265] Step 3: The generative AI adjusts the generation of digital assets according to the user's emotions. For example, when the user feels happy, the generative AI generates digital assets with bright tones. Conversely, when the user feels sad, the generative AI generates digital assets with dark tones.

[0266] "Example 2"

[0267] Step 1: The emotion engine recognizes the user's emotions. Specifically, it estimates emotions from the user's facial expressions, tone of voice, text input, etc.

[0268] Step 2: Generative AI generates digital assets.

[0269] Step 3: The digital world (metaverse) populates with the generated digital assets.

[0270] Step 4: The digital world (metaverse) adjusts the placement of digital assets according to the user's emotions. For example, when the user feels happy, the digital world (metaverse) places digital assets with bright colors around the user. Conversely, when the user feels sad, the digital world (metaverse) places digital assets with dark colors around the user.

[0271] Example 1

[0272] Next, a description will be given of Example 1 of the embodiment 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal".

[0273] In conventional digital asset generation systems, it was difficult to generate digital assets according to the user's emotions, and it was not possible to increase user satisfaction. In addition, there was a lack of means to effectively place the generated digital assets in the digital world and improve the user's experience.

[0274] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0275] In this invention, the server includes a means for generating digital assets using generative AI, a means for placing the generated digital assets in the digital world, a means for recognizing a user's emotions, a means for feeding back the recognized emotion data to the generative AI and adjusting the generation of the digital assets, and a means for expanding the digital world. This enables the generation of digital assets according to the user's emotions, thereby increasing the user's satisfaction. In addition, it is possible to effectively place the generated digital assets in the digital world, improving the user's experience.

[0276] "Generative AI" is an artificial intelligence technology for automatically generating digital assets.

[0277] "Digital Asset" means an asset or object that exists in digital form, such as an image, video, audio, or 3D model.

[0278] "Digital world" refers to computer-generated virtual spaces, including virtual reality and augmented reality.

[0279] "Means for recognizing user emotions" refers to technology that estimates emotions by analyzing a user's facial expressions, tone of voice, text input, etc.

[0280] "Emotion data" is data that indicates the user's emotional state, and includes emotions such as "happiness," "sadness," and "excitement," for example.

[0281] "Feedback" is the process of returning information to the system to adjust its next actions or processing based on the information it has obtained.

[0282] "Means of expanding the digital world" are technologies that increase the content and functionality of a virtual space by adding digital assets.

[0283] This invention uses a system that combines generative AI and an emotion engine to generate digital assets according to the user's emotions and place them in the digital world. A specific embodiment of this system will be described below.

[0284] The server generates a program that combines a generative AI and an emotion engine. This program has the function of generating digital assets and adjusting the generation according to the user's emotions. As the generative AI, for example, models such as DALL-E and GPT-3 (registered trademark) from OpenAI can be used. As the emotion engine, for example, the emotion recognition API of Azure (registered trademark) Cognitive Services from Microsoft (registered trademark) can be used.

[0285] A user requests the generation of a digital asset using a device such as a smartphone or PC. For example, the user enters the text "I want to create a new avatar." The device uses the built-in camera and microphone to capture the user's facial expressions and tone of voice, which are then analyzed by the emotion engine. For example, the camera detects the user's smile, and the microphone analyzes a bright tone of voice.

[0286] The terminal sends the analysis result to the server as data that "the user is happy." Based on the received emotion data, the server sends a prompt to the generative AI saying, "The user is happy, so please generate a bright and colorful avatar." The generative AI generates a bright and colorful avatar based on the prompt. For example, using DALL-E, an avatar wearing colorful clothes is generated.

[0287] The server places the generated avatar as an avatar in the user's metaverse. For example, the server places a new avatar in the home area in the user's metaverse. The user logs into the metaverse through a terminal and checks the new avatar. The user is satisfied with the generated avatar.

[0288] As a concrete example, consider a user who wants to create a new avatar in the metaverse. The user types the text "I want to create a new avatar" into the device. The emotion engine recognizes from the user's input and facial expression that the user is excited. The server receives this information and instructs the generative AI to generate a bright and colorful avatar.

[0289] Example of a prompt to be input to a generative AI model:

[0290] "Users want to create new avatars. Users are excited. Generate bright, colorful avatars."

[0291] In this way, digital assets corresponding to the user's emotions are generated and placed in the digital world, thereby increasing the user's satisfaction.

[0292] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0293] Step 1:

[0294] A user requests the creation of a digital asset.

[0295] Users use a device such as a smartphone or PC to open a dedicated application and enter the text, "I would like to create a new avatar."

[0296] Input: User text input (e.g. "I would like to create a new avatar")

[0297] Output: User request data

[0298] Step 2:

[0299] The terminal recognizes the user's emotions.

[0300] The device uses the built-in camera and microphone to capture the user's facial expressions and tone of voice, which are then analyzed by the emotion engine. For example, the camera can detect a user's smile, and the microphone can detect a bright tone of voice.

[0301] Input: User's facial expression data, voice tone data

[0302] Output: User emotion data (e.g. "The user is happy")

[0303] Step 3:

[0304] The terminal transmits the emotion data to the server.

[0305] The terminal transmits the analysis result to the server as data indicating that "the user is happy."

[0306] Input: User emotion data

[0307] Output: Send emotion data to the server

[0308] Step 4:

[0309] The server receives the emotion data and sends prompts to the generative AI.

[0310] Based on the emotional data it receives, the server sends a prompt to the generative AI saying, "The user is happy, so please generate a bright and colorful avatar."

[0311] Input: User emotion data

[0312] Output: A prompt for the generative AI (e.g., "The user is happy, so please generate a bright and colorful avatar.")

[0313] Step 5:

[0314] Generative AI generates digital assets.

[0315] The generative AI will generate bright and colorful avatars based on prompts, for example using DALL-E to generate an avatar wearing colorful clothing.

[0316] Input: Prompt statement

[0317] Output: Generated digital assets (e.g. bright and colorful avatars)

[0318] Step 6:

[0319] The server places the generated digital assets in the metaverse.

[0320] The server places the generated avatar as an avatar in the user's metaverse, for example by placing the new avatar in a home area in the user's metaverse.

[0321] Input: Generated digital assets

[0322] Output: Digital assets placed in the metaverse

[0323] Step 7:

[0324] The user reviews the generated digital assets.

[0325] The user logs into the metaverse through the terminal and checks the new avatar. The user is satisfied with the generated avatar.

[0326] Input: Digital assets placed within the metaverse

[0327] Output: User confirmation and satisfaction

[0328] (Application example 1)

[0329] Next, a description will be given of Application Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0330] Conventional digital asset generation systems do not provide personalization according to the user's emotions, and the user experience is uniform. In addition, the shopping experience in the virtual store is uniform, and there is also the issue that it is not possible to flexibly respond to the user's emotions. This leads to a decrease in user satisfaction, and it is difficult to promote the use of virtual stores.

[0331] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0332] In this invention, the server includes a means for generating digital assets using generative AI, a means for arranging the generated digital assets in a digital world (metaverse), a means for adjusting the generation of digital assets according to the user's emotions using an emotion engine that recognizes the user's emotions, a means for exploring a virtual store using a smartphone, and a means for generating and arranging personalized digital assets based on the emotions, and a means for expanding the digital world (metaverse). This enables a personalized shopping experience according to the user's emotions.

[0333] "Generative AI" is an artificial intelligence technology for automatically generating digital assets.

[0334] "Digital assets" refers to assets or content that exist in digital form, including, for example, NFTs.

[0335] The "digital world (metaverse)" refers to a digital space that includes virtual reality and augmented reality, and is an environment that users can experience interactively.

[0336] An "emotion engine" is a technology for recognizing and estimating emotions from a user's facial expressions, tone of voice, text input, etc.

[0337] A "virtual store" is a virtual store that exists in the digital world (metaverse) where users can browse and purchase products.

[0338] A "personalized shopping experience" refers to a shopping experience that is customized according to a user's individual feelings and preferences.

[0339] A "smartphone" is a mobile phone that can connect to the Internet and run applications.

[0340] "NFT" stands for Non-Fungible Token, a token that uses blockchain technology to prove ownership of digital assets.

[0341] The system for implementing the present invention uses generative AI to generate digital assets and arranges the digital assets personalized according to the user's emotions in a virtual store. A specific embodiment of this system will be described below.

[0342] System Configuration

[0343] The system consists of the following main components:

[0344] 1. Generative AI: Artificial intelligence technology for automatically generating digital assets.

[0345] 2. Emotion engine: Technology that recognizes and estimates emotions from the user's facial expressions, tone of voice, text input, etc.

[0346] 3. Smartphone: A mobile phone that can connect to the Internet and run applications.

[0347] 4. Virtual Store: A virtual store that exists within the digital world (metaverse).

[0348] 5. Metaverse API: An API for placing digital assets within the metaverse.

[0349] Program Processing

[0350] The server captures the user's image using the smartphone camera and inputs it into the emotion engine. The emotion engine recognizes the user's emotion in real time and feeds the result back to the generative AI. The generative AI generates digital assets according to the user's emotion and places them in the virtual store using the metaverse API.

[0351] Hardware and Software Used

[0352] Hardware: Smartphone camera

[0353] software:

[0354] OpenCV: Camera image capture and processing

[0355] EmotionEngine: An emotion engine for recognizing user emotions

[0356] AssetGenerator: A generative AI that generates digital assets based on emotions

[0357] MetaverseAPI: API for placing digital assets in the metaverse

[0358] Examples

[0359] For example, while a user is exploring a virtual store using their smartphone, the camera captures the user's smile. The emotion engine recognizes this smile as "joy" and the generative AI generates bright and colorful digital assets (e.g. special promotional products or decorations). The metaverse API places this digital asset in a specific location in the virtual store, allowing the user to enjoy a shopping experience that matches their emotions.

[0360] Examples of prompt statements

[0361] "A prompt for a digital asset to generate when the user is delighted: 'Generate bright and colorful promotional merchandise.'"

[0362] In this way, a personalized shopping experience can be provided according to the user's emotions.

[0363] The flow of the specific process in the application example 1 will be described with reference to FIG.

[0364] Step 1:

[0365] The server captures the user's video using the smartphone camera. The input is the smartphone camera video and the output is the captured video data, which is used for subsequent emotion recognition processing.

[0366] Step 2:

[0367] The server inputs the captured video data into an emotion engine to recognize the user's emotion. The input is the captured video data, and the output is the recognized emotion data. The emotion engine analyzes the user's facial expressions from the video data and estimates the emotion.

[0368] Step 3:

[0369] The server feeds back the recognized emotion data to the generative AI and instructs it to generate a digital asset. The input is the recognized emotion data, and the output is the generated digital asset. The generative AI generates a prompt sentence based on the emotion data, and generates the digital asset according to the prompt sentence.

[0370] Step 4:

[0371] The server inputs the generated digital asset to the metaverse API and places it in the virtual store. The input is the generated digital asset, and the output is the digital asset placed in the virtual store. The metaverse API places the digital asset in the specified location.

[0372] Step 5:

[0373] A user uses a smartphone to explore the virtual store and check the placed digital assets. The input is the digital assets in the virtual store, and the output is the user's visual experience. The user can enjoy a personalized shopping experience according to his / her emotions.

[0374] Example 2

[0375] Next, a second embodiment of the second embodiment will be described. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0376] In the conventional digital world (metaverse), it was difficult to dynamically generate and place digital assets according to the user's emotions. In addition, there was a lack of means to expand the digital world based on the user's emotions, so the improvement of the user experience was limited. This caused a problem of a decrease in the immersion and satisfaction that users felt in the digital world.

[0377] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0378] In this invention, the server includes a means for generating digital assets using generative AI, a means for arranging the generated digital assets in the digital world, a means for collecting user emotion data, a means for adjusting the arrangement of the digital assets based on the collected emotion data, and a means for expanding the digital world. This enables dynamic generation and arrangement of digital assets according to the user's emotions, thereby improving the user experience and expanding the digital world.

[0379] "Generative AI" is an artificial intelligence technology for automatically generating digital assets.

[0380] "Digital Asset" refers to a virtual object or environmental element used in the digital world.

[0381] "Digital world" refers to computer-generated virtual environments, including virtual reality and augmented reality.

[0382] "User emotion data" is data indicating the user's emotional state obtained from the user's facial expression, voice, etc.

[0383] An "emotion engine" is software or hardware for analyzing a user's emotion data and recognizing the user's emotional state.

[0384] The "means for adjusting the placement" refers to a function for dynamically changing the placement of digital assets based on the user's emotion data.

[0385] "Means for expanding the digital world" refers to functionality for expanding the scope and content of the digital world using generated digital assets.

[0386] DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0387] The present invention is a system that uses generative AI to generate digital assets and arranges and adjusts them in the digital world based on the user's emotional data. A specific embodiment of the system will be described below.

[0388] Hardware and Software Used

[0389] Server: A computer system with high-performance computing power that is used to run the generative AI model. Specifically, we use OpenAI's GPT-4 as the generative AI model.

[0390] Terminal: A device equipped with a camera and microphone to collect user emotion data. Affectiva's emotion recognition software is used as the emotion engine.

[0391] Digital World (Metaverse): A computer-generated virtual environment, including virtual reality and augmented reality, that is a platform for the placement of generated digital assets.

[0392] Program Processing

[0393] 1. Digital Asset Creation

[0394] The server generates digital assets using a generative AI model. For example, the server inputs a prompt sentence such as "Generate a new fantasy castle" to the generative AI and generates a digital asset of a castle.

[0395] 2. Collecting user emotion data

[0396] The device collects the user's facial expressions and voice in real time. The device's camera captures the user's face and the microphone records the user's voice. This data is sent to the emotion engine.

[0397] 3. Emotion Data Analysis

[0398] The device uses an emotion engine to analyze the collected emotion data and recognize the user's emotional state. For example, the device analyzes the user's facial expressions and voice and determines that the user is happy.

[0399] 4. Placing and adjusting digital assets

[0400] The digital world (metaverse) arranges the generated digital assets according to the user's emotions. For example, when the user is happy, the metaverse arranges bright digital assets (e.g., bright flowers and light effects) around the user. Conversely, when the user is sad, it arranges dark digital assets (e.g., dark trees and shadow effects).

[0401] Examples

[0402] Example prompt:

[0403] "Generate a new fantasy castle"

[0404] "Generate the right digital assets when users are happy"

[0405] "Generate appropriate digital assets when users are sad"

[0406] This system collects user emotion data in real time, generates digital assets using a generative AI model based on that data, and optimally places them according to the user's emotions, thereby improving the user experience.

[0407] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0408] Step 1:

[0409] Collecting user emotion data

[0410] The device collects the user's facial expressions and voice in real time.

[0411] Input: The device's camera captures the user's face and the microphone records the user's voice.

[0412] Data processing: The collected facial expression and voice data is sent to the emotion engine, where pre-processing is performed to analyze the user's emotional state.

[0413] Output: Preprocessed emotion data is generated.

[0414] How it works: The device's camera captures the user's face and the microphone records the user's voice, and these data are sent to the emotion engine in real time.

[0415] Step 2:

[0416] Emotion data analysis

[0417] The terminal uses an emotion engine to analyze the collected emotion data and recognize the user's emotional state.

[0418] Input: Preprocessed emotion data.

[0419] Data calculation: The emotion engine analyzes facial expression data and voice data to recognize the user's emotional state (e.g., happiness, sadness, surprise, etc.).

[0420] Output: Data indicating the user's emotional state.

[0421] Specific operation: The emotion engine analyzes the user's facial expressions and voice and determines that the user is happy.

[0422] Step 3:

[0423] Creating digital assets

[0424] The server generates digital assets using a generative AI model.

[0425] Input: Data describing the user's emotional state and a prompt.

[0426] Data calculation: The server sends a prompt sentence to the generative AI model and generates a digital asset according to the user's emotional state.

[0427] Output: The generated digital assets.

[0428] Specific operation: The server sends a prompt sentence to the generative AI model, "Generate a digital asset that is appropriate when the user is happy," and generates a digital asset with a bright color tone.

[0429] Step 4:

[0430] Placing and adjusting digital assets

[0431] The digital world (metaverse) will place the generated digital assets according to the emotions of users.

[0432] Input: The generated digital asset and data indicating the user's emotional state.

[0433] Data Computation: The digital world dynamically adjusts the placement of digital assets based on the user's emotional state.

[0434] Output: Placement of digital assets according to user emotions.

[0435] Specific behavior: When the user is happy, the metaverse will place generated digital assets in light tones (e.g., bright flowers and light effects) around the user. Conversely, when the user is sad, the metaverse will place dark digital assets (e.g., dark trees and shadow effects).

[0436] Step 5:

[0437] User feedback

[0438] The terminal displays the arranged digital assets to the user and again collects the user's reaction.

[0439] Input: The placed digital asset.

[0440] Data processing: User responses are collected and pre-processed for use in the next cycle.

[0441] Output: Preprocessed user response data.

[0442] Specific operation: The generated digital asset is displayed on the device display, and the user looks at it and expresses emotion again. This emotion data is collected again and used for the next cycle.

[0443] (Application example 2)

[0444] Next, a description will be given of Application Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0445] In conventional metaverse systems, it was difficult to dynamically change the environment according to the user's emotions, and the improvement of the user experience was limited. In addition, in virtual stores, product suggestions and adjustments to the store's atmosphere based on the user's emotions were not made, making it difficult to provide an optimal shopping experience for each individual user. This resulted in problems such as a decrease in user satisfaction and engagement.

[0446] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0447] In this invention, the server includes a means for generating digital assets using generative AI, a means for arranging the generated digital assets in a digital world (metaverse), a means for expanding the digital world (metaverse), a means for adjusting the arrangement of the digital assets according to the user's emotions using an emotion engine that recognizes the user's emotions, and a means for dynamically changing the atmosphere and product suggestions in a virtual store using an application installed on a smartphone. This makes it possible to dynamically change the environment according to the user's emotions, and to provide each user with an optimal shopping experience.

[0448] "Generative AI" is an artificial intelligence technology that automatically generates digital assets.

[0449] A "digital asset" is a virtual object or resource used within the digital world (metaverse).

[0450] The "digital world (metaverse)" is a virtual space or environment constructed on the Internet.

[0451] An "emotion engine" is a technology for recognizing emotions from a user's facial expressions, voice, etc.

[0452] A "smartphone" is a portable information terminal that can run a variety of applications in addition to the functions of a mobile phone.

[0453] An "application" is a software program designed to provide a particular function or service.

[0454] A "virtual store" is a virtual shopping space that exists on the Internet.

[0455] "Atmosphere" is the overall feeling or impression of a particular environment or space.

[0456] "Product suggestion" is the act of recommending a particular product to a user.

[0457] "Dynamic change" means changing in real time according to the situation or conditions.

[0458] The system for implementing the present invention includes a generative AI, an emotion engine, an application installed on a smartphone, and a metaverse platform. A detailed description of each component is provided below.

[0459] Generative AI

[0460] Generative AI is an artificial intelligence technology that automatically generates digital assets. The server uses generative AI to generate digital assets according to the user's emotions. For example, if the user is happy, the server generates digital assets in bright tones, and if the user is sad, the server generates digital assets in dark tones.

[0461] Emotion Engine

[0462] The emotion engine is a technology for recognizing emotions from a user's facial expressions, voice, etc. The smartphone's camera and microphone are used to collect the user's emotion data, which the emotion engine analyzes to recognize the user's emotion.

[0463] Smartphone Application

[0464] The smartphone application dynamically changes the store layout and products using digital assets generated by generative AI while the user is shopping in the virtual store. The application receives emotion data from the emotion engine and adjusts the store's atmosphere and product suggestions based on that data.

[0465] Metaverse Platform

[0466] The metaverse platform is a platform for placing generated digital assets and expanding the digital world (metaverse). The server places the digital assets generated by generative AI on the metaverse platform and adjusts the placement according to the user's emotions.

[0467] Examples

[0468] When a user visits a virtual store and smiles through the camera, the emotion engine recognizes it as "happy." This causes the generative AI to generate bright-toned digital assets and place them on the metaverse platform. Users can enjoy shopping in a bright and fun atmosphere.

[0469] Examples of prompt statements

[0470] "When the user smiles, generate a brightly colored digital asset and place it in the virtual store."

[0471] In this way, it becomes possible to dynamically change the environment in response to the user's emotions, and to provide each individual user with an optimal shopping experience.

[0472] The flow of the specific process in the application example 2 will be described with reference to FIG.

[0473] Step 1:

[0474] The user provides emotion data via the smartphone's camera and microphone.

[0475] Input: User's facial expressions and voice

[0476] Output: Emotion data (image data and audio data)

[0477] Specific actions: A user launches a virtual store application, smiles at the camera, and speaks.

[0478] Step 2:

[0479] The terminal uses an emotion engine to analyze the emotion data and recognize the user's emotion.

[0480] Input: Emotion data (image data or audio data)

[0481] Output: User emotion (e.g. happy, sad)

[0482] Specific operation: The smartphone application sends data obtained from the camera and microphone to the emotion engine, which then analyzes the data and recognizes the user's emotions.

[0483] Step 3:

[0484] The server uses generative AI to generate digital assets based on the recognized emotions.

[0485] Input: User emotion (e.g. happy, sad)

[0486] Output: Digital Assets (e.g. light objects, dark objects)

[0487] Specific operation: Based on the emotion data received by the server from the emotion engine, the generative AI is activated and a digital asset with an appropriate theme is generated.

[0488] Step 4:

[0489] The server places the generated digital assets on the metaverse platform.

[0490] Input: Digital assets (e.g. light-toned objects, dark-toned objects)

[0491] Output: Placed digital assets

[0492] Specific operation: The server sends the digital assets received from the generative AI to the metaverse platform and places them in the specified location.

[0493] Step 5:

[0494] The terminal receives data from the metaverse platform and dynamically changes the atmosphere and product suggestions within the virtual store.

[0495] Input: Placed digital assets

[0496] Output: Modified virtual store layout and product suggestions

[0497] How it works: A smartphone application receives data from the metaverse platform and updates the user interface to show the user new moods and product suggestions.

[0498] In this way, it becomes possible to dynamically change the environment in response to the user's emotions, and to provide each individual user with an optimal shopping experience.

[0499] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires a voice indicating a user input for the result of the specific processing. The control unit 46A transmits the voice data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[0500] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (registered trademark) (Internet search<URL: https: / / openai.com / blog / chatgpt> ) and other generation AIs. The data generation model 58 is obtained by making a neural network perform deep learning. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating a voice, text data indicating a text, and image data indicating an image is input. The data generation model 58 performs inference on the input inference data according to the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0501] Another example of generative AI is Gemini (registered trademark) (Internet search <url: https: gemini.google.com ?hl="ja">) are mentioned.

[0502] In the above embodiment, an example was given in which the specific process was performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0503] [Second embodiment]

[0504] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0505] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0506] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a wide area network (WAN) and / or a local area network (LAN).

[0507] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0508] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs the voice according to instructions from the processor 46.

[0509] Camera 42 is a small digital camera equipped with an optical system including a lens, an aperture, and a shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (e.g., an imaging range defined by an angle of view equivalent to the width of the field of vision of an average healthy person).

[0510] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for transmitting and receiving various types of information between the processor 46 and the processor 28 via the network 54. The transmission and reception of various types of information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is performed in a secure state.

[0511] Fig. 4 shows an example of main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0512] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32, and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0513] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0514] In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50, and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0515] Next, the specific processing by the specific processing unit 290 of the data processing device 12 will be described.

[0516] "Example 1"

[0517] The present invention is a system for generating digital assets using a generative AI. Specifically, the generative AI mass-produces digital assets such as NFTs. The generated digital assets are placed in the digital world (metaverse). For example, the digital assets generated by the generative AI are placed as specific locations or objects in the metaverse.

[0518] "Example 2"

[0519] Furthermore, an embodiment of the present invention is a system for expanding the digital world (metaverse). Specifically, the digital world (metaverse) functions as a platform that can fully utilize digital assets generated by generative AI. For example, the metaverse creates and expands new regions, buildings, and objects using digital assets generated by generative AI.

[0520] The process flow of each embodiment will be described below.

[0521] "Example 1"

[0522] Step 1: Generative AI generates digital assets such as NFTs. This generation process is based on specific algorithms and rules.

[0523] Step 2: The generated digital assets are placed in the digital world (metaverse). Placement is done as a specific place or object. For example, the digital assets are placed as part of a building or scenery in the metaverse.

[0524] "Example 2"

[0525] Step 1: The digital world (metaverse) will function as a platform where digital assets generated by generative AI can be fully utilized.

[0526] Step 2: The metaverse creates and expands new regions, buildings, and objects using digital assets generated by the generative AI, for example to develop new regions and improve existing ones.

[0527] Example 1

[0528] Next, a description will be given of Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal".

[0529] Conventional digital asset generation systems have a problem in that the process for users to generate specific digital assets is complicated, and it is time-consuming to place the generated digital assets in the metaverse. In addition, there is a lack of a platform for efficiently mass-producing the generated digital assets and utilizing them in the metaverse.

[0530] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0531] In this invention, the server includes a means for a user to input a prompt sentence, a means for a terminal to send a prompt sentence to the server, a means for the server to analyze the prompt sentence, a means for the server to input a prompt sentence to a generative AI model, a means for the generative AI model to generate a digital asset, a means for the server to receive the generated digital asset, a means for the server to place the digital asset in a digital world (metaverse), and a means for the user to confirm the generated digital asset in the digital world (metaverse). This allows the user to simply input a prompt sentence to generate various digital assets using the generative AI model and place them in the metaverse.

[0532] "User" means an individual or entity that uses the system to generate and verify Digital Assets within the Metaverse.

[0533] A "prompt sentence" is text that describes the specifications and characteristics of the digital asset that the user wants to generate.

[0534] A "terminal" is an electronic device that a user uses to enter prompt text and send it to a server.

[0535] A "server" is a computer system that parses prompt statements, inputs them into the generative AI model, receives the generated digital assets, and places them in the metaverse.

[0536] A “generative AI model” is an artificial intelligence model that generates digital assets based on a prompt sentence.

[0537] "Digital Asset" means an asset in digital form, such as image data or a 3D model, that is generated by a generative AI model.

[0538] The "metaverse" is a virtual space in which digital assets are located and which users can access and use.

[0539] DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0540] The present invention is a system for generating digital assets using a generative AI model and placing the digital assets in the metaverse. A specific embodiment of the system will be described below.

[0541] 1. System Configuration

[0542] The system consists of a terminal where the user inputs a prompt sentence, a server that analyzes the prompt sentence and inputs it into the generative AI model, and infrastructure for placing the generated digital assets in the metaverse.

[0543] 2. Hardware and Software Used

[0544] Terminal: A device through which a user enters a prompt, such as a computer, smartphone, or tablet.

[0545] Server: A computer system for parsing prompts and inputting them into a generative AI model. The server parses the prompts using a natural language processing (NLP) library (e.g., spaCy or NLTK).

[0546] Generative AI model: An artificial intelligence model that generates digital assets based on a prompt, such as OpenAI's GPT-4 or DALL-E.

[0547] Metaverse: A virtual space where generated digital assets are placed. Digital assets are placed using the metaverse API.

[0548] 3. Data processing and calculation

[0549] Prompt analysis: The server analyzes the prompt entered by the user using natural language processing technology. For example, the server extracts the keyword "blue dragon" from the prompt "Please generate an NFT of a blue dragon."

[0550] Digital asset generation: The generative AI model generates digital assets based on the parsed prompt sentences. The generated digital assets are output as image data or 3D models.

[0551] Placing digital assets: The server places the generated digital assets in a specific location in the metaverse. Using the metaverse API, the server places the digital assets in the virtual space.

[0552] 4. Specific Examples

[0553] The user inputs a prompt sentence saying "I want you to generate a red phoenix NFT." The terminal sends this prompt sentence to the server. The server parses the prompt sentence and extracts the keyword "red phoenix." The server then inputs this keyword into the generative AI model and generates a red phoenix digital asset. The generated digital asset is placed in a specific area in the metaverse by the server. The user can access the metaverse and check the generated red phoenix NFT.

[0554] In this way, users can simply enter a prompt to have a generative AI model generate various digital assets and place them in the metaverse.

[0555] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0556] Step 1:

[0557] The user enters a prompt sentence.

[0558] Specifically, the user enters a prompt such as "Please generate an NFT of a blue dragon" into the device's input form.

[0559] Input: Prompt text (e.g. "I want you to generate an NFT of a blue dragon")

[0560] Output: The prompt text entered in the terminal

[0561] Step 2:

[0562] The terminal sends a prompt to the server.

[0563] In concrete terms, when the user clicks the send button, the terminal sends a prompt text to the server using an HTTP request.

[0564] Input: The prompt text entered into the terminal

[0565] Output: The prompt text sent to the server.

[0566] Step 3:

[0567] The server parses the prompt.

[0568] Specifically, the server uses a natural language processing (NLP) library (e.g., spaCy or NLTK) to parse the prompt and extract the keyword "blue dragon."

[0569] Input: The prompt text sent to the server

[0570] Output: Parsed keywords (e.g. "blue dragon")

[0571] Step 4:

[0572] The server inputs a prompt sentence into the generative AI model.

[0573] Specifically, the server sends an API request to input the analyzed keywords into a generative AI model (e.g., GPT-4 or DALL-E).

[0574] Input: Parsed keyword

[0575] Output: The request sent to the generative AI model

[0576] Step 5:

[0577] A generative AI model generates digital assets.

[0578] Specifically, the generative AI model runs an image generation algorithm internally to generate the "blue dragon" digital asset.

[0579] Input: The request sent to the generative AI model

[0580] Output: Generated digital assets (e.g. image data of a blue dragon)

[0581] Step 6:

[0582] The server receives the generated digital asset.

[0583] Specifically, the server receives the digital assets returned from the generative AI model and stores them in a database.

[0584] Input: Generated digital assets

[0585] Output: Digital assets stored in a database

[0586] Step 7:

[0587] The server places the digital assets in the metaverse.

[0588] Specifically, the server uses the metaverse's API to place the generated digital assets in specific locations in the virtual space.

[0589] Input: Digital assets stored in a database

[0590] Output: Digital assets placed in the metaverse

[0591] Step 8:

[0592] The user views the generated digital assets in the metaverse.

[0593] Specifically, users will access the metaverse using a VR headset or PC and view the placed blue dragon NFT.

[0594] Input: Digital assets placed within the metaverse

[0595] Output: Digital assets viewed by the user

[0596] (Application example 1)

[0597] Next, a description will be given of Application Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal".

[0598] In conventional digital asset generation systems, it was difficult for users to easily generate and place digital assets suitable for specific locations in a virtual store. In addition, there was a lack of means to effectively utilize the generated digital assets in the metaverse. This resulted in a problem that virtual store owners could not quickly create and place their own digital assets, making store operations in the metaverse inefficient.

[0599] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0600] In this invention, the server includes a means for generating digital assets using generative AI, a means for arranging the generated digital assets in a digital world (metaverse), a means for expanding the digital world (metaverse), a means for generating digital assets based on a prompt sentence input by a user, and a means for arranging the generated digital assets in a specific location in a virtual store. This makes it possible for a user to generate and quickly arrange suitable digital assets in a virtual store simply by inputting a prompt sentence.

[0601] "Generative AI" is an artificial intelligence system that automatically generates digital assets based on prompts entered by users.

[0602] A "digital asset" is any digital form of property used within the digital world (metaverse), including, for example, NFTs.

[0603] The "digital world (metaverse)" is a virtual space built on the Internet, and is a platform where users can utilize digital assets to take action.

[0604] A "prompt" is text that a user inputs into a generative AI that dictates the content of the digital asset to be generated.

[0605] A "virtual store" is a virtual store that exists in the digital world (metaverse) and is a place where users can purchase goods and services.

[0606] A "means of placement" is a technical means for placing a generated digital asset at a specific location within the digital world (metaverse).

[0607] The following system configuration will be described as an embodiment of the present invention.

[0608] First, the server generates a digital asset using a generative AI. A generative AI is an artificial intelligence system that automatically generates a digital asset based on a prompt sentence entered by a user. A user enters a prompt sentence using a device such as a smartphone, smart glasses, or a head-mounted display. For example, by entering the prompt sentence "a luxurious virtual store entrance," the generative AI generates a digital asset based on that instruction.

[0609] The generated digital assets are placed in the digital world (metaverse) by the server. The digital world (metaverse) is a virtual space built on the Internet and is a platform where users can use digital assets to take part in activities. The server provides the technical means to place the generated digital assets in specific locations within the virtual store.

[0610] Specifically, the server does the following:

[0611] 1. Receive the prompt text entered by the user.

[0612] 2. Use generative AI to generate digital assets based on the prompt.

[0613] 3. Place the resulting digital asset in a designated location within the metaverse.

[0614] The implementation of this system uses the following hardware and software:

[0615] Hardware: Smartphones, smart glasses, head-mounted displays

[0616] Software: OpenAI API, Metaverse API, Python

[0617] As a specific example, when a user inputs the prompt sentence "A luxurious virtual store entrance," the generative AI generates a luxurious digital asset based on that instruction. The generated digital asset is placed at the entrance of the virtual store by the server. This allows the user to generate and quickly place a suitable digital asset in a virtual store simply by inputting a prompt sentence.

[0618] The flow of the specific process in the application example 1 will be described with reference to FIG.

[0619] Step 1:

[0620] The user enters a prompt sentence at the terminal.

[0621] Specifically, the user inputs a prompt sentence indicating the content of the digital asset to be generated using a terminal such as a smartphone, smart glasses, or a head-mounted display. For example, the user inputs the prompt sentence "a luxurious virtual store entrance."

[0622] Input: Prompt text (e.g. "A luxurious virtual store entrance")

[0623] Output: The prompt is sent to the server.

[0624] Step 2:

[0625] The server receives the prompt text and sends it to the generative AI.

[0626] Specifically, the server receives a prompt sentence sent by a user and sends the prompt sentence to the generative AI, which then starts a process to generate a digital asset based on the prompt sentence.

[0627] Input: prompt statement

[0628] Output: The prompt is sent to the generative AI.

[0629] Step 3:

[0630] A generative AI generates digital assets based on the prompt text.

[0631] Specifically, the generative AI analyzes the prompt text and generates an appropriate digital asset based on its content. For example, based on the prompt text "A high-end virtual store entrance," it generates a high-end digital asset.

[0632] Input: prompt statement

[0633] Output: Generated digital assets

[0634] Step 4:

[0635] The server receives the generated digital assets and places them in specific locations within the digital world (metaverse) using the metaverse API.

[0636] Specifically, the server receives the digital assets generated by the generative AI and uses the metaverse API to place the digital assets in a specified location in the virtual store. For example, a luxurious digital asset is placed at the entrance of the virtual store.

[0637] Input: Generated digital assets, location information

[0638] Output: A digital asset is placed in a specific location in the metaverse.

[0639] Step 5:

[0640] A user uses a terminal to check a virtual store in the metaverse.

[0641] Specifically, users use devices such as smartphones, smart glasses, and head-mounted displays to check the virtual store in the metaverse and verify that the generated digital assets are correctly placed.

[0642] Input: Virtual store information in the metaverse

[0643] Output: User confirms placement of digital assets

[0644] Example 2

[0645] Next, a description will be given of Example 2 of the embodiment 2. In the following description, the data processing device 12 is referred to as a "server" and the smart glasses 214 are referred to as a "terminal".

[0646] In the conventional digital world (metaverse), the means by which users could generate and place new digital assets were limited, making it difficult to expand the metaverse. In addition, there was a lack of efficient methods for managing and placing the generated digital assets. This made it difficult for users to maximize their creativity, and restricted the development of the metaverse.

[0647] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0648] In this invention, the server includes a means for a user to input a prompt sentence, a means for the server to receive the prompt sentence and input it to the generative AI model, a means for the server to generate a digital asset using the generative AI model, a means for the server to store the generated digital asset in a database, a means for a terminal to obtain the digital asset from the server, a means for a user to place the digital asset in a digital world (metaverse) using the terminal, and a means for expanding the digital world (metaverse). This allows a user to efficiently generate digital assets using the generative AI model and freely place them in the metaverse.

[0649] A "User" is an individual or entity that utilizes the system to enter prompt statements and place generated digital assets within the metaverse.

[0650] A "prompt sentence" is a textual instruction sentence that a user inputs to instruct a generative AI model to generate a digital asset.

[0651] "Server" means a computer system for receiving prompt sentences, generating digital assets using a generative AI model, and storing them in a database.

[0652] A "generative AI model" is an artificial intelligence model for generating digital assets based on prompt sentences entered by a user.

[0653] "Digital assets" are digital content such as 3D models, textures, and animations that are generated by generative AI models and used within the metaverse.

[0654] "Database" means an information management system for storing generated digital assets and their metadata.

[0655] A "terminal" is hardware such as a computer or VR device that a user uses to obtain digital assets from the server and place them in the metaverse.

[0656] The "metaverse" is a virtual space where generated digital assets can be placed and where users can interact with each other.

[0657] A "means for placement" is a method or tool that allows a user to place a digital asset at a specific location in the metaverse using a terminal.

[0658] An "expansion tool" is a method or tool for increasing areas and objects within the metaverse and expanding the virtual space using generated digital assets.

[0659] The present invention is a system for generating digital assets using a generative AI model and placing them in a digital world (metaverse). A specific embodiment of the system will be described below.

[0660] First, the user inputs a prompt on the Metaverse interface. The prompt instructs the generative AI model on the content of the digital asset they want to generate. For example, they input a prompt such as "I want you to generate a 3D model of a futuristic city."

[0661] Next, the server receives the prompt sentence sent by the user. The received prompt sentence is input to the generative AI model. For example, a general artificial intelligence model (such as GPT-4 or DALL-E) can be used as the generative AI model.

[0662] The server uses a generative AI model to generate digital assets based on the prompt text. The generated digital assets can range from 3D models, textures, animations, and more. The generated digital assets are stored in a database on the server. The database also stores metadata for the generated digital assets (such as the generation date and time, the prompt text, etc.).

[0663] After that, the terminal obtains the digital asset from the server in response to the user's request. The obtained digital asset is temporarily stored in the terminal's local storage. The terminal can be the user's PC or VR device.

[0664] Users place digital assets acquired using their devices in the metaverse. For example, a 3D model of a futuristic city can be placed as a new area. In this case, users can intuitively operate the assets using a VR device or PC.

[0665] For example, if a user inputs the prompt "I want you to generate a 3D model of a futuristic city," the server uses the generative AI model to generate a 3D model of a futuristic city. This 3D model is stored on the server and sent to the user's device. The user can place this 3D model in the metaverse and use it as a new region.

[0666] Examples of prompts include the following:

[0667] "I want you to generate a 3D model of a medieval castle."

[0668] "I want you to generate a 3D model of a futuristic city."

[0669] "I want you to generate a fantasy forest texture."

[0670] This system allows users to efficiently generate digital assets using generative AI models and place them freely in the metaverse, making it easier to expand the metaverse and maximizing users' creativity.

[0671] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0672] Step 1:

[0673] The user enters a prompt sentence.

[0674] A user enters "I want you to generate a 3D model of a futuristic city" into a text box on the Metaverse interface and clicks the send button. The entered prompt is sent to the server.

[0675] Step 2:

[0676] The server receives the prompt sentence and inputs it into the generative AI model.

[0677] The server receives the prompt text sent by the user. It sends the received prompt text as a request to the API of the generative AI model. The input is the prompt text, and the output is the digital asset data generated by the generative AI model.

[0678] Step 3:

[0679] The server generates the digital assets using the generative AI model.

[0680] The server receives the digital asset data returned by the generative AI model and converts it into a 3D model file format (e.g., OBJ or FBX). The input is the data from the generative AI model, and the output is the converted 3D model file.

[0681] Step 4:

[0682] The server stores the generated digital assets in a database.

[0683] The server saves the generated 3D model to a database and logs the saving completion status. The input is the 3D model file and the output is the saving completion status to the database.

[0684] Step 5:

[0685] The terminal obtains the digital asset from the server.

[0686] The terminal sends a request to the server in response to the user's operation and downloads the generated 3D model. After the download is complete, it is saved in local storage. The input is the request to the server, and the output is the 3D model file saved in local storage.

[0687] Step 6:

[0688] A user places a digital asset in the metaverse using a terminal.

[0689] The user puts on the VR device and switches to edit mode in the metaverse. The user selects the acquired 3D model and places it in the desired position by dragging and dropping it. After placing it, the user clicks the save button to confirm the changes. The input is the 3D model file stored in the local storage, and the output is the 3D model placed in the metaverse.

[0690] (Application example 2)

[0691] Next, a description will be given of Application Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server", and the smart glasses 214 will be referred to as a "terminal".

[0692] In conventional metaverse platforms, the process for users to generate and deploy digital assets is complicated and requires specialized knowledge, making it difficult for general users to use. In addition, in virtual stores, it is difficult to freely customize store layouts and product displays, preventing users from fully expressing their creativity.

[0693] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0694] In this invention, the server includes a means for generating digital assets using a generative AI, a means for placing the generated digital assets in a digital world (metaverse), a means for expanding the digital world (metaverse), a means for the generative AI to generate digital assets in response to a user inputting a prompt sentence, and a means for placing the generated digital assets in a virtual store. This allows a user to generate digital assets by simply inputting a prompt sentence without requiring specialized knowledge, and to freely place them in a virtual store.

[0695] "Generative AI" is artificial intelligence that automatically generates digital assets based on prompts entered by users.

[0696] "Digital assets" is a general term for virtual objects and resources used within the digital world (metaverse).

[0697] The "digital world (metaverse)" is a virtual space built on the Internet, and is a platform where users can interact and engage in activities through avatars.

[0698] A "prompt sentence" is text that a user enters to instruct the generative AI on the content of the digital asset they want to generate.

[0699] A "virtual store" is a virtual store that exists in the digital world (metaverse) where users can browse and purchase products.

[0700] A "means of placement" is a method or technology for placing a generated digital asset at a specific location within the digital world (metaverse).

[0701] "Augmentation tools" are methods and technologies for expanding the scope and content of the digital world (the metaverse).

[0702] The system for implementing this invention includes a means for generating digital assets using a generative AI, placing them in a digital world (metaverse), and further expanding the digital world. Specifically, the generative AI generates digital assets when a user inputs a prompt sentence, and the generated digital assets can be placed in a virtual store.

[0703] 1. System Programming

[0704] The system uses the following hardware and software:

[0705] Hardware: Smartphones, smart glasses, head-mounted displays

[0706] Software: OpenAI API, Unreal Engine

[0707] 2. Explanation of program processing

[0708] The server receives the prompt text entered by the user and sends it to the generative AI. The generative AI generates a digital asset based on the prompt text and returns the data to the server. The server reads the returned digital asset data using Unreal Engine and places it in the specified location.

[0709] The specific process is as follows:

[0710] 1. The user uses a smartphone or head-mounted display to input a prompt describing the digital asset they want to generate.

[0711] 2. The server sends this prompt to the OpenAI API, and the generative AI generates the digital asset data.

[0712] 3. The generated digital asset data is returned to the server, which uses the Unreal Engine to load it and place it in the specified location within the virtual store.

[0713] 3. Specific Examples

[0714] For example, if the user types "I want to create a new shelf", the prompt text might look like this:

[0715] Example prompt:

[0716] "Generate a new product shelf. The shelf is made of wood and has 5 shelves, with different types of products on each shelf."

[0717] When this prompt is sent to the generative AI, the AI ​​generates a five-tier wooden shelf with different types of products on each shelf, and returns a digital asset that the server can then place in a virtual store.

[0718] In this way, users can generate digital assets by simply entering prompt sentences without the need for specialized knowledge, and place them freely in a virtual store.

[0719] The flow of the specific process in the application example 2 will be described with reference to FIG.

[0720] Step 1:

[0721] The user uses a smartphone or a head-mounted display to input a description of the digital asset to be generated as a prompt sentence.

[0722] Input: A prompt to be entered by the user (e.g. "Create a new shelf. The shelf is made of wood and has five shelves, each with a different type of product on it.")

[0723] Output: The prompt text is sent to the server.

[0724] Step 2:

[0725] The server sends the received prompt text to the OpenAI API, and the generative AI generates data for the digital asset.

[0726] Input: The prompt received by the server

[0727] Output: Digital asset data generated by generative AI

[0728] Specific operation: The server sends the prompt text to the OpenAI API, and the generative AI generates a digital asset based on the prompt text.

[0729] Step 3:

[0730] The generated digital asset data is returned to the server.

[0731] Input: Digital asset data generated by generative AI

[0732] Output: The digital asset data is returned to the server.

[0733] Specific operation: The generative AI returns the generated digital asset data to the server.

[0734] Step 4:

[0735] The server loads the returned digital asset data using the Unreal Engine and places it in the specified location within the virtual store.

[0736] Input: The digital asset's data returned by the server

[0737] Output: Digital assets are placed in a virtual store.

[0738] Specific operation: The server uses Unreal Engine to load the digital asset data and place it in the specified location in the virtual store.

[0739] Step 5:

[0740] Users can view the digital assets placed in the virtual store through their smartphones or head-mounted displays.

[0741] Input: Digital assets placed in a virtual store

[0742] Output: A representation of the digital asset for the user to see

[0743] Specific operation: The user uses a smartphone or a head-mounted display to visually check the digital assets placed in the virtual store.

[0744] In addition, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0745] "Example 1"

[0746] In one embodiment of the present invention, a generative AI and an emotion engine are combined. The generative AI generates digital assets. Meanwhile, the emotion engine recognizes the user's emotions. Specifically, the emotion is estimated from the user's facial expressions, tone of voice, text input, and the like. The output of this emotion engine is fed back to the generative AI, and the generation of the digital asset is adjusted according to the user's emotions. For example, when the user feels joy, the generative AI generates a digital asset with a bright color tone. Conversely, when the user feels sad, the generative AI generates a digital asset with a dark color tone.

[0747] "Example 2"

[0748] In another embodiment of the present invention, a generative AI, an emotion engine, and a digital world (metaverse) are combined. The generative AI generates digital assets, and the emotion engine recognizes the user's emotions. The digital world (metaverse) then arranges the generated digital assets and adjusts the arrangement according to the user's emotions. For example, when a user feels joy, the digital world (metaverse) arranges digital assets in bright tones around the user. Conversely, when a user feels sad, the digital world (metaverse) arranges digital assets in dark tones around the user.

[0749] The process flow of each embodiment will be described below.

[0750] "Example 1"

[0751] Step 1: The emotion engine recognizes the user's emotions. Specifically, it estimates emotions from the user's facial expressions, tone of voice, text input, etc.

[0752] Step 2: The output of the emotion engine is fed back to the generative AI.

[0753] Step 3: The generative AI adjusts the generation of digital assets according to the user's emotions. For example, when the user feels happy, the generative AI generates digital assets with bright tones. Conversely, when the user feels sad, the generative AI generates digital assets with dark tones.

[0754] Generate a set of

[0755] "Example 2"

[0756] Step 1: The emotion engine recognizes the user's emotions. Specifically, it estimates emotions from the user's facial expressions, tone of voice, text input, etc.

[0757] Step 2: Generative AI generates digital assets.

[0758] Step 3: The digital world (metaverse) populates with the generated digital assets.

[0759] Step 4: The digital world (metaverse) adjusts the placement of digital assets according to the user's emotions. For example, when the user feels happy, the digital world (metaverse) places digital assets with bright colors around the user. Conversely, when the user feels sad, the digital world (metaverse) places digital assets with dark colors around the user.

[0760] Example 1

[0761] Next, a description will be given of Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal".

[0762] In conventional digital asset generation systems, it was difficult to generate digital assets according to the user's emotions, and it was not possible to increase user satisfaction. In addition, there was a lack of means to effectively place the generated digital assets in the digital world and improve the user's experience.

[0763] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0764] In this invention, the server includes a means for generating digital assets using generative AI, a means for placing the generated digital assets in the digital world, a means for recognizing a user's emotions, a means for feeding back the recognized emotion data to the generative AI and adjusting the generation of the digital assets, and a means for expanding the digital world. This enables the generation of digital assets according to the user's emotions, thereby increasing the user's satisfaction. In addition, it is possible to effectively place the generated digital assets in the digital world, improving the user's experience.

[0765] "Generative AI" is an artificial intelligence technology for automatically generating digital assets.

[0766] "Digital Asset" means an asset or object that exists in digital form, such as an image, video, audio, or 3D model.

[0767] "Digital world" refers to computer-generated virtual spaces, including virtual reality and augmented reality.

[0768] "Means for recognizing user emotions" refers to technology that estimates emotions by analyzing a user's facial expressions, tone of voice, text input, etc.

[0769] "Emotion data" is data that indicates the user's emotional state, and includes emotions such as "happiness," "sadness," and "excitement," for example.

[0770] "Feedback" is the process of returning information to the system to adjust its next actions or processing based on the information it has obtained.

[0771] "Means of expanding the digital world" are technologies that increase the content and functionality of a virtual space by adding digital assets.

[0772] This invention uses a system that combines generative AI and an emotion engine to generate digital assets according to the user's emotions and place them in the digital world. A specific embodiment of this system will be described below.

[0773] The server generates a program that combines a generative AI and an emotion engine. This program has the function of generating digital assets and adjusting the generation according to the user's emotions. For example, models such as OpenAI's DALL-E and GPT-3 can be used as generative AI. For example, the emotion recognition API of Microsoft's Azure Cognitive Services can be used as the emotion engine.

[0774] A user requests the generation of a digital asset using a device such as a smartphone or PC. For example, the user enters the text "I want to create a new avatar." The device uses the built-in camera and microphone to capture the user's facial expressions and tone of voice, which are then analyzed by the emotion engine. For example, the camera detects the user's smile, and the microphone analyzes a bright tone of voice.

[0775] The terminal sends the analysis result to the server as data that "the user is happy." Based on the received emotion data, the server sends a prompt to the generative AI saying, "The user is happy, so please generate a bright and colorful avatar." The generative AI generates a bright and colorful avatar based on the prompt. For example, using DALL-E, an avatar wearing colorful clothes is generated.

[0776] The server places the generated avatar as an avatar in the user's metaverse. For example, the server places a new avatar in the home area in the user's metaverse. The user logs into the metaverse through a terminal and checks the new avatar. The user is satisfied with the generated avatar.

[0777] As a concrete example, consider a user who wants to create a new avatar in the metaverse. The user types the text "I want to create a new avatar" into the device. The emotion engine recognizes from the user's input and facial expression that the user is excited. The server receives this information and instructs the generative AI to generate a bright and colorful avatar.

[0778] Example of a prompt to be input to a generative AI model:

[0779] "Users want to create new avatars. Users are excited. Generate bright, colorful avatars."

[0780] In this way, digital assets corresponding to the user's emotions are generated and placed in the digital world, thereby increasing the user's satisfaction.

[0781] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0782] Step 1:

[0783] A user requests the creation of a digital asset.

[0784] Users use a device such as a smartphone or PC to open a dedicated application and enter the text, "I would like to create a new avatar."

[0785] Input: User text input (e.g. "I would like to create a new avatar")

[0786] Output: User request data

[0787] Step 2:

[0788] The terminal recognizes the user's emotions.

[0789] The device uses the built-in camera and microphone to capture the user's facial expressions and tone of voice, which are then analyzed by the emotion engine. For example, the camera can detect a user's smile, and the microphone can detect a bright tone of voice.

[0790] Input: User's facial expression data, voice tone data

[0791] Output: User emotion data (e.g. "The user is happy")

[0792] Step 3:

[0793] The terminal transmits the emotion data to the server.

[0794] The terminal transmits the analysis result to the server as data indicating that "the user is happy."

[0795] Input: User emotion data

[0796] Output: Send emotion data to the server

[0797] Step 4:

[0798] The server receives the emotion data and sends prompts to the generative AI.

[0799] Based on the emotional data it receives, the server sends a prompt to the generative AI saying, "The user is happy, so please generate a bright and colorful avatar."

[0800] Input: User emotion data

[0801] Output: A prompt for the generative AI (e.g., "The user is happy, so please generate a bright and colorful avatar.")

[0802] Step 5:

[0803] Generative AI generates digital assets.

[0804] The generative AI will generate bright and colorful avatars based on prompts, for example using DALL-E to generate an avatar wearing colorful clothing.

[0805] Input: Prompt statement

[0806] Output: Generated digital assets (e.g. bright and colorful avatars)

[0807] Step 6:

[0808] The server places the generated digital assets in the metaverse.

[0809] The server places the generated avatar as an avatar in the user's metaverse, for example by placing the new avatar in a home area in the user's metaverse.

[0810] Input: Generated digital assets

[0811] Output: Digital assets placed in the metaverse

[0812] Step 7:

[0813] The user reviews the generated digital assets.

[0814] The user logs into the metaverse through the terminal and checks the new avatar. The user is satisfied with the generated avatar.

[0815] Input: Digital assets placed within the metaverse

[0816] Output: User confirmation and satisfaction

[0817] (Application example 1)

[0818] Next, a description will be given of Application Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal".

[0819] Conventional digital asset generation systems do not provide personalization according to the user's emotions, and the user experience is uniform. In addition, the shopping experience in the virtual store is uniform, and there is also the issue that it is not possible to flexibly respond to the user's emotions. This leads to a decrease in user satisfaction, and it is difficult to promote the use of virtual stores.

[0820] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0821] In this invention, the server includes a means for generating digital assets using generative AI, a means for arranging the generated digital assets in a digital world (metaverse), a means for adjusting the generation of digital assets according to the user's emotions using an emotion engine that recognizes the user's emotions, a means for exploring a virtual store using a smartphone, and a means for generating and arranging personalized digital assets based on the emotions, and a means for expanding the digital world (metaverse). This enables a personalized shopping experience according to the user's emotions.

[0822] "Generative AI" is an artificial intelligence technology for automatically generating digital assets.

[0823] "Digital assets" refers to assets or content that exist in digital form, including, for example, NFTs.

[0824] The "digital world (metaverse)" refers to a digital space that includes virtual reality and augmented reality, and is an environment that users can experience interactively.

[0825] An "emotion engine" is a technology for recognizing and estimating emotions from a user's facial expressions, tone of voice, text input, etc.

[0826] A "virtual store" is a virtual store that exists in the digital world (metaverse) where users can browse and purchase products.

[0827] A "personalized shopping experience" refers to a shopping experience that is customized according to a user's individual feelings and preferences.

[0828] A "smartphone" is a mobile phone that can connect to the Internet and run applications.

[0829] "NFT" stands for Non-Fungible Token, a token that uses blockchain technology to prove ownership of digital assets.

[0830] The system for implementing the present invention uses generative AI to generate digital assets and arranges the digital assets personalized according to the user's emotions in a virtual store. A specific embodiment of this system will be described below.

[0831] System Configuration

[0832] The system consists of the following main components:

[0833] 1. Generative AI: Artificial intelligence technology for automatically generating digital assets.

[0834] 2. Emotion engine: Technology that recognizes and estimates emotions from the user's facial expressions, tone of voice, text input, etc.

[0835] 3. Smartphone: A mobile phone that can connect to the Internet and run applications.

[0836] 4. Virtual Store: A virtual store that exists within the digital world (metaverse).

[0837] 5. Metaverse API: An API for placing digital assets within the metaverse.

[0838] Program Processing

[0839] The server captures the user's image using the smartphone camera and inputs it into the emotion engine. The emotion engine recognizes the user's emotion in real time and feeds the result back to the generative AI. The generative AI generates digital assets according to the user's emotion and places them in the virtual store using the metaverse API.

[0840] Hardware and Software Used

[0841] Hardware: Smartphone camera

[0842] software:

[0843] OpenCV: Camera image capture and processing

[0844] EmotionEngine: An emotion engine for recognizing user emotions

[0845] AssetGenerator: A generative AI that generates digital assets based on emotions

[0846] MetaverseAPI: API for placing digital assets in the metaverse

[0847] Examples

[0848] For example, while a user is exploring a virtual store using their smartphone, the camera captures the user's smile. The emotion engine recognizes this smile as "joy" and the generative AI generates bright and colorful digital assets (e.g. special promotional products or decorations). The metaverse API places this digital asset in a specific location in the virtual store, allowing the user to enjoy a shopping experience that matches their emotions.

[0849] Examples of prompt statements

[0850] "A prompt for a digital asset to generate when the user is delighted: 'Generate bright and colorful promotional merchandise.'"

[0851] In this way, a personalized shopping experience can be provided according to the user's emotions.

[0852] The flow of the specific process in the application example 1 will be described with reference to FIG.

[0853] Step 1:

[0854] The server captures the user's video using the smartphone camera. The input is the smartphone camera video and the output is the captured video data, which is used for subsequent emotion recognition processing.

[0855] Step 2:

[0856] The server inputs the captured video data into an emotion engine to recognize the user's emotion. The input is the captured video data, and the output is the recognized emotion data. The emotion engine analyzes the user's facial expressions from the video data and estimates the emotion.

[0857] Step 3:

[0858] The server feeds back the recognized emotion data to the generative AI and instructs it to generate a digital asset. The input is the recognized emotion data, and the output is the generated digital asset. The generative AI generates a prompt sentence based on the emotion data, and generates the digital asset according to the prompt sentence.

[0859] Step 4:

[0860] The server inputs the generated digital asset to the metaverse API and places it in the virtual store. The input is the generated digital asset, and the output is the digital asset placed in the virtual store. The metaverse API places the digital asset in the specified location.

[0861] Step 5:

[0862] A user uses a smartphone to explore the virtual store and check the placed digital assets. The input is the digital assets in the virtual store, and the output is the user's visual experience. The user can enjoy a personalized shopping experience according to his / her emotions.

[0863] Example 2

[0864] Next, a description will be given of Example 2 of the embodiment 2. In the following description, the data processing device 12 is referred to as a "server" and the smart glasses 214 are referred to as a "terminal".

[0865] In the conventional digital world (metaverse), it was difficult to dynamically generate and place digital assets according to the user's emotions. In addition, there was a lack of means to expand the digital world based on the user's emotions, so the improvement of the user experience was limited. This caused a problem of a decrease in the immersion and satisfaction that users felt in the digital world.

[0866] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0867] In this invention, the server includes a means for generating digital assets using generative AI, a means for arranging the generated digital assets in the digital world, a means for collecting user emotion data, a means for adjusting the arrangement of the digital assets based on the collected emotion data, and a means for expanding the digital world. This enables dynamic generation and arrangement of digital assets according to the user's emotions, thereby improving the user experience and expanding the digital world.

[0868] "Generative AI" is an artificial intelligence technology for automatically generating digital assets.

[0869] "Digital Asset" refers to a virtual object or environmental element used in the digital world.

[0870] "Digital world" refers to computer-generated virtual environments, including virtual reality and augmented reality.

[0871] "User emotion data" is data indicating the user's emotional state obtained from the user's facial expression, voice, etc.

[0872] An "emotion engine" is software or hardware for analyzing a user's emotion data and recognizing the user's emotional state.

[0873] The "means for adjusting the placement" refers to a function for dynamically changing the placement of digital assets based on the user's emotion data.

[0874] "Means for expanding the digital world" refers to functionality for expanding the scope and content of the digital world using generated digital assets.

[0875] DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0876] The present invention is a system that uses generative AI to generate digital assets and arranges and adjusts them in the digital world based on the user's emotional data. A specific embodiment of the system will be described below.

[0877] Hardware and Software Used

[0878] Server: A computer system with high-performance computing power that is used to run the generative AI model. Specifically, we use OpenAI's GPT-4 as the generative AI model.

[0879] Terminal: A device equipped with a camera and microphone to collect user emotion data. Affectiva's emotion recognition software is used as the emotion engine.

[0880] Digital World (Metaverse): A computer-generated virtual environment, including virtual reality and augmented reality, that is a platform for the placement of generated digital assets.

[0881] Program Processing

[0882] 1. Digital Asset Creation

[0883] The server generates digital assets using a generative AI model. For example, the server inputs a prompt sentence such as "Generate a new fantasy castle" to the generative AI and generates a digital asset of a castle.

[0884] 2. Collecting user emotion data

[0885] The device collects the user's facial expressions and voice in real time. The device's camera captures the user's face and the microphone records the user's voice. This data is sent to the emotion engine.

[0886] 3. Emotion Data Analysis

[0887] The device uses an emotion engine to analyze the collected emotion data and recognize the user's emotional state. For example, the device analyzes the user's facial expressions and voice and determines that the user is happy.

[0888] 4. Placing and adjusting digital assets

[0889] The digital world (metaverse) arranges the generated digital assets according to the user's emotions. For example, when the user is happy, the metaverse arranges bright digital assets (e.g., bright flowers and light effects) around the user. Conversely, when the user is sad, it arranges dark digital assets (e.g., dark trees and shadow effects).

[0890] Examples

[0891] Example prompt:

[0892] "Generate a new fantasy castle"

[0893] "Generate the right digital assets when users are happy"

[0894] "Generate appropriate digital assets when users are sad"

[0895] This system collects user emotion data in real time, generates digital assets using a generative AI model based on that data, and optimally places them according to the user's emotions, thereby improving the user experience.

[0896] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0897] Step 1:

[0898] Collecting user emotion data

[0899] The device collects the user's facial expressions and voice in real time.

[0900] Input: The device's camera captures the user's face and the microphone records the user's voice.

[0901] Data processing: The collected facial expression and voice data is sent to the emotion engine, where pre-processing is performed to analyze the user's emotional state.

[0902] Output: Preprocessed emotion data is generated.

[0903] How it works: The device's camera captures the user's face and the microphone records the user's voice, and these data are sent to the emotion engine in real time.

[0904] Step 2:

[0905] Emotion data analysis

[0906] The terminal uses an emotion engine to analyze the collected emotion data and recognize the user's emotional state.

[0907] Input: Preprocessed emotion data.

[0908] Data calculation: The emotion engine analyzes facial expression data and voice data to recognize the user's emotional state (e.g., happiness, sadness, surprise, etc.).

[0909] Output: Data indicating the user's emotional state.

[0910] Specific operation: The emotion engine analyzes the user's facial expressions and voice and determines that the user is happy.

[0911] Step 3:

[0912] Creating digital assets

[0913] The server generates digital assets using a generative AI model.

[0914] Input: Data describing the user's emotional state and a prompt.

[0915] Data calculation: The server sends a prompt sentence to the generative AI model and generates a digital asset according to the user's emotional state.

[0916] Output: The generated digital assets.

[0917] Specific operation: The server sends a prompt sentence to the generative AI model, "Generate a digital asset that is appropriate when the user is happy," and generates a digital asset with a bright color tone.

[0918] Step 4:

[0919] Placing and adjusting digital assets

[0920] The digital world (metaverse) will place the generated digital assets according to the emotions of users.

[0921] Input: The generated digital asset and data indicating the user's emotional state.

[0922] Data Computation: The digital world dynamically adjusts the placement of digital assets based on the user's emotional state.

[0923] Output: Placement of digital assets according to user emotions.

[0924] Specific behavior: When the user is happy, the metaverse will place generated digital assets in light tones (e.g., bright flowers and light effects) around the user. Conversely, when the user is sad, the metaverse will place dark digital assets (e.g., dark trees and shadow effects).

[0925] Step 5:

[0926] User feedback

[0927] The terminal displays the arranged digital assets to the user and again collects the user's reaction.

[0928] Input: The placed digital asset.

[0929] Data processing: User responses are collected and pre-processed for use in the next cycle.

[0930] Output: Preprocessed user response data.

[0931] Specific operation: The generated digital asset is displayed on the device display, and the user looks at it and expresses emotion again. This emotion data is collected again and used for the next cycle.

[0932] (Application example 2)

[0933] Next, a description will be given of Application Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server", and the smart glasses 214 will be referred to as a "terminal".

[0934] In conventional metaverse systems, it was difficult to dynamically change the environment according to the user's emotions, and the improvement of the user experience was limited. In addition, in virtual stores, product suggestions and adjustments to the store's atmosphere based on the user's emotions were not made, making it difficult to provide an optimal shopping experience for each individual user. This resulted in problems such as a decrease in user satisfaction and engagement.

[0935] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0936] In this invention, the server includes a means for generating digital assets using generative AI, a means for arranging the generated digital assets in a digital world (metaverse), a means for expanding the digital world (metaverse), a means for adjusting the arrangement of the digital assets according to the user's emotions using an emotion engine that recognizes the user's emotions, and a means for dynamically changing the atmosphere and product suggestions in a virtual store using an application installed on a smartphone. This makes it possible to dynamically change the environment according to the user's emotions, and to provide each user with an optimal shopping experience.

[0937] "Generative AI" is an artificial intelligence technology that automatically generates digital assets.

[0938] A "digital asset" is a virtual object or resource used within the digital world (metaverse).

[0939] The "digital world (metaverse)" is a virtual space or environment constructed on the Internet.

[0940] An "emotion engine" is a technology for recognizing emotions from a user's facial expressions, voice, etc.

[0941] A "smartphone" is a portable information terminal that can run a variety of applications in addition to the functions of a mobile phone.

[0942] An "Application" is a software program designed to provide a particular function or service.

[0943] A "virtual store" is a virtual shopping space that exists on the Internet.

[0944] "Atmosphere" is the overall feeling or impression of a particular environment or space.

[0945] "Product suggestion" is the act of recommending a particular product to a user.

[0946] "Dynamic change" means changing in real time according to the situation or conditions.

[0947] The system for implementing the present invention includes a generative AI, an emotion engine, an application installed on a smartphone, and a metaverse platform. A detailed description of each component is provided below.

[0948] Generative AI

[0949] Generative AI is an artificial intelligence technology that automatically generates digital assets. The server uses generative AI to generate digital assets according to the user's emotions. For example, if the user is happy, the server generates digital assets in bright tones, and if the user is sad, the server generates digital assets in dark tones.

[0950] Emotion Engine

[0951] The emotion engine is a technology for recognizing emotions from a user's facial expressions, voice, etc. The smartphone's camera and microphone are used to collect the user's emotion data, which the emotion engine analyzes to recognize the user's emotion.

[0952] Smartphone Application

[0953] The smartphone application dynamically changes the store layout and products using digital assets generated by generative AI while the user is shopping in the virtual store. The application receives emotion data from the emotion engine and adjusts the store's atmosphere and product suggestions based on that data.

[0954] Metaverse Platform

[0955] The metaverse platform is a platform for placing generated digital assets and expanding the digital world (metaverse). The server places the digital assets generated by generative AI on the metaverse platform and adjusts the placement according to the user's emotions.

[0956] Examples

[0957] When a user visits a virtual store and smiles through the camera, the emotion engine recognizes it as "happy." This causes the generative AI to generate bright-toned digital assets and place them on the metaverse platform. Users can enjoy shopping in a bright and fun atmosphere.

[0958] Examples of prompt statements

[0959] "When the user smiles, generate a brightly colored digital asset and place it in the virtual store."

[0960] In this way, it becomes possible to dynamically change the environment in response to the user's emotions, and to provide each individual user with an optimal shopping experience.

[0961] The flow of the specific process in the application example 2 will be described with reference to FIG.

[0962] Step 1:

[0963] The user provides emotion data via the smartphone's camera and microphone.

[0964] Input: User's facial expressions and voice

[0965] Output: Emotion data (image data and audio data)

[0966] Specific actions: A user launches a virtual store application, smiles at the camera, and speaks.

[0967] Step 2:

[0968] The terminal uses an emotion engine to analyze the emotion data and recognize the user's emotion.

[0969] Input: Emotion data (image data or audio data)

[0970] Output: User's emotion (e.g. happy, sad)

[0971] Specific operation: The smartphone application sends data obtained from the camera and microphone to the emotion engine, which then analyzes the data and recognizes the user's emotions.

[0972] Step 3:

[0973] The server uses generative AI to generate digital assets based on the recognized emotions.

[0974] Input: User emotion (e.g. happy, sad)

[0975] Output: Digital Assets (e.g. light objects, dark objects)

[0976] Specific operation: Based on the emotion data received by the server from the emotion engine, the generative AI is activated and a digital asset with an appropriate theme is generated.

[0977] Step 4:

[0978] The server places the generated digital assets on the metaverse platform.

[0979] Input: Digital assets (e.g. light-toned objects, dark-toned objects)

[0980] Output: Placed digital assets

[0981] Specific operation: The server sends the digital assets received from the generative AI to the metaverse platform and places them in the specified location.

[0982] Step 5:

[0983] The terminal receives data from the metaverse platform and dynamically changes the atmosphere and product suggestions within the virtual store.

[0984] Input: Placed digital assets

[0985] Output: Modified virtual store layout and product suggestions

[0986] How it works: A smartphone application receives data from the metaverse platform and updates the user interface to show the user new moods and product suggestions.

[0987] In this way, it becomes possible to dynamically change the environment in response to the user's emotions, and to provide each individual user with an optimal shopping experience.

[0988] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires a voice indicating a user input for the result of the specific processing. The control unit 46A transmits the voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[0989] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ) and other generation AIs. The data generation model 58 is obtained by making a neural network perform deep learning. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating a voice, text data indicating a text, and image data indicating an image is input. The data generation model 58 performs inference on the input inference data according to the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0990] Another example of generative AI is Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) are mentioned.

[0991] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[0992] [Third embodiment]

[0993] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[0994] 5, the data processing system 310 includes the data processing device 12 and a headset type terminal 314. An example of the data processing device 12 is a server.

[0995] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a wide area network (WAN) and / or a local area network (LAN).

[0996] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[0997] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs the voice according to instructions from the processor 46.

[0998] Camera 42 is a small digital camera equipped with an optical system including a lens, an aperture, and a shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (e.g., an imaging range defined by an angle of view equivalent to the width of the field of vision of an average healthy person).

[0999] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for transmitting and receiving various types of information between the processor 46 and the processor 28 via the network 54. The transmission and reception of various types of information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is performed in a secure state.

[1000] Fig. 6 shows an example of main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1001] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32, and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1002] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1003] In the headset type terminal 314, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50, and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1004] Next, the specific processing by the specific processing unit 290 of the data processing device 12 will be described.

[1005] "Example 1"

[1006] The present invention is a system for generating digital assets using a generative AI. Specifically, the generative AI mass-produces digital assets such as NFTs. The generated digital assets are placed in the digital world (metaverse). For example, the digital assets generated by the generative AI are placed as specific locations or objects in the metaverse.

[1007] "Example 2"

[1008] Furthermore, an embodiment of the present invention is a system for expanding the digital world (metaverse). Specifically, the digital world (metaverse) functions as a platform that can fully utilize digital assets generated by generative AI. For example, the metaverse creates and expands new regions, buildings, and objects using digital assets generated by generative AI.

[1009] The process flow of each embodiment will be described below.

[1010] "Example 1"

[1011] Step 1: Generative AI generates digital assets such as NFTs. This generation process is based on specific algorithms and rules.

[1012] Step 2: The generated digital assets are placed in the digital world (metaverse). Placement is done as a specific place or object. For example, the digital assets are placed as part of a building or scenery in the metaverse.

[1013] "Example 2"

[1014] Step 1: The digital world (metaverse) will function as a platform where digital assets generated by generative AI can be fully utilized.

[1015] Step 2: The metaverse creates and expands new regions, buildings, and objects using digital assets generated by the generative AI, for example to develop new regions and improve existing ones.

[1016] Example 1

[1017] Next, a description will be given of Example 1 of the embodiment 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal".

[1018] Conventional digital asset generation systems have a problem in that the process for users to generate specific digital assets is complicated, and it is time-consuming to place the generated digital assets in the metaverse. In addition, there is a lack of a platform for efficiently mass-producing the generated digital assets and utilizing them in the metaverse.

[1019] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1020] In this invention, the server includes a means for a user to input a prompt sentence, a means for a terminal to send a prompt sentence to the server, a means for the server to analyze the prompt sentence, a means for the server to input a prompt sentence to a generative AI model, a means for the generative AI model to generate a digital asset, a means for the server to receive the generated digital asset, a means for the server to place the digital asset in a digital world (metaverse), and a means for the user to confirm the generated digital asset in the digital world (metaverse). This allows the user to simply input a prompt sentence to generate various digital assets using the generative AI model and place them in the metaverse.

[1021] "User" means an individual or entity that uses the system to generate and verify Digital Assets within the Metaverse.

[1022] A "prompt sentence" is text that describes the specifications and characteristics of the digital asset that the user wants to generate.

[1023] A "terminal" is an electronic device that a user uses to enter prompt text and send it to a server.

[1024] A "server" is a computer system that parses prompt statements, inputs them into the generative AI model, receives the generated digital assets, and places them in the metaverse.

[1025] A “generative AI model” is an artificial intelligence model that generates digital assets based on a prompt sentence.

[1026] "Digital Asset" means an asset in digital form, such as image data or a 3D model, that is generated by a generative AI model.

[1027] The "metaverse" is a virtual space in which digital assets are located and which users can access and use.

[1028] DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[1029] The present invention is a system for generating digital assets using a generative AI model and placing the digital assets in the metaverse. A specific embodiment of the system will be described below.

[1030] 1. System Configuration

[1031] The system consists of a terminal where the user inputs a prompt sentence, a server that analyzes the prompt sentence and inputs it into the generative AI model, and infrastructure for placing the generated digital assets in the metaverse.

[1032] 2. Hardware and Software Used

[1033] Terminal: A device through which a user enters a prompt, such as a computer, smartphone, or tablet.

[1034] Server: A computer system for parsing prompts and inputting them into a generative AI model. The server parses the prompts using a natural language processing (NLP) library (e.g., spaCy or NLTK).

[1035] Generative AI model: An artificial intelligence model that generates digital assets based on a prompt, such as OpenAI's GPT-4 or DALL-E.

[1036] Metaverse: A virtual space where generated digital assets are placed. Digital assets are placed using the metaverse API.

[1037] 3. Data processing and calculation

[1038] Prompt analysis: The server analyzes the prompt entered by the user using natural language processing technology. For example, the server extracts the keyword "blue dragon" from the prompt "Please generate an NFT of a blue dragon."

[1039] Digital asset generation: The generative AI model generates digital assets based on the parsed prompt sentences. The generated digital assets are output as image data or 3D models.

[1040] Placing digital assets: The server places the generated digital assets in a specific location in the metaverse. Using the metaverse API, the server places the digital assets in the virtual space.

[1041] 4. Specific Examples

[1042] The user inputs a prompt sentence saying "I want you to generate a red phoenix NFT." The terminal sends this prompt sentence to the server. The server parses the prompt sentence and extracts the keyword "red phoenix." The server then inputs this keyword into the generative AI model and generates a red phoenix digital asset. The generated digital asset is placed in a specific area in the metaverse by the server. The user can access the metaverse and check the generated red phoenix NFT.

[1043] In this way, users can simply enter a prompt to have a generative AI model generate various digital assets and place them in the metaverse.

[1044] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1045] Step 1:

[1046] The user enters a prompt sentence.

[1047] Specifically, the user enters a prompt such as "Please generate an NFT of a blue dragon" into the device's input form.

[1048] Input: Prompt text (e.g. "I want you to generate an NFT of a blue dragon")

[1049] Output: The prompt text entered in the terminal

[1050] Step 2:

[1051] The terminal sends a prompt to the server.

[1052] In concrete terms, when the user clicks the send button, the terminal sends a prompt text to the server using an HTTP request.

[1053] Input: The prompt text entered into the terminal

[1054] Output: The prompt text sent to the server.

[1055] Step 3:

[1056] The server parses the prompt.

[1057] Specifically, the server uses a natural language processing (NLP) library (e.g., spaCy or NLTK) to parse the prompt and extract the keyword "blue dragon."

[1058] Input: The prompt text sent to the server

[1059] Output: Parsed keywords (e.g. "blue dragon")

[1060] Step 4:

[1061] The server inputs a prompt sentence into the generative AI model.

[1062] Specifically, the server sends an API request to input the analyzed keywords into a generative AI model (e.g., GPT-4 or DALL-E).

[1063] Input: Parsed keyword

[1064] Output: The request sent to the generative AI model

[1065] Step 5:

[1066] A generative AI model generates digital assets.

[1067] Specifically, the generative AI model runs an image generation algorithm internally to generate the "blue dragon" digital asset.

[1068] Input: The request sent to the generative AI model

[1069] Output: Generated digital assets (e.g. image data of a blue dragon)

[1070] Step 6:

[1071] The server receives the generated digital asset.

[1072] Specifically, the server receives the digital assets returned from the generative AI model and stores them in a database.

[1073] Input: Generated digital assets

[1074] Output: Digital assets stored in a database

[1075] Step 7:

[1076] The server places the digital assets in the metaverse.

[1077] Specifically, the server uses the metaverse's API to place the generated digital assets in specific locations in the virtual space.

[1078] Input: Digital assets stored in a database

[1079] Output: Digital assets placed in the metaverse

[1080] Step 8:

[1081] The user views the generated digital assets in the metaverse.

[1082] Specifically, users will access the metaverse using a VR headset or PC and view the placed blue dragon NFT.

[1083] Input: Digital assets placed within the metaverse

[1084] Output: Digital assets viewed by the user

[1085] (Application example 1)

[1086] Next, a description will be given of Application Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server", and the headset type terminal 314 will be referred to as a "terminal".

[1087] In conventional digital asset generation systems, it was difficult for users to easily generate and place digital assets suitable for specific locations in a virtual store. In addition, there was a lack of means to effectively utilize the generated digital assets in the metaverse. This resulted in a problem that virtual store owners could not quickly create and place their own digital assets, making store operations in the metaverse inefficient.

[1088] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1089] In this invention, the server includes a means for generating digital assets using generative AI, a means for arranging the generated digital assets in a digital world (metaverse), a means for expanding the digital world (metaverse), a means for generating digital assets based on a prompt sentence input by a user, and a means for arranging the generated digital assets in a specific location in a virtual store. This makes it possible for a user to generate and quickly arrange suitable digital assets in a virtual store simply by inputting a prompt sentence.

[1090] "Generative AI" is an artificial intelligence system that automatically generates digital assets based on prompts entered by users.

[1091] A "digital asset" is any digital form of property used within the digital world (metaverse), including, for example, NFTs.

[1092] The "digital world (metaverse)" is a virtual space built on the Internet, and is a platform where users can utilize digital assets to take action.

[1093] A "prompt" is text that a user inputs into a generative AI that dictates the content of the digital asset to be generated.

[1094] A "virtual store" is a virtual store that exists in the digital world (metaverse) and is a place where users can purchase goods and services.

[1095] A "means of placement" is a technical means for placing a generated digital asset at a specific location within the digital world (metaverse).

[1096] The following system configuration will be described as an embodiment of the present invention.

[1097] First, the server generates a digital asset using a generative AI. A generative AI is an artificial intelligence system that automatically generates a digital asset based on a prompt sentence entered by a user. A user enters a prompt sentence using a device such as a smartphone, smart glasses, or a head-mounted display. For example, by entering the prompt sentence "a luxurious virtual store entrance," the generative AI generates a digital asset based on that instruction.

[1098] The generated digital assets are placed in the digital world (metaverse) by the server. The digital world (metaverse) is a virtual space built on the Internet and is a platform where users can use digital assets to take part in activities. The server provides the technical means to place the generated digital assets in specific locations within the virtual store.

[1099] Specifically, the server does the following:

[1100] 1. Receive the prompt text entered by the user.

[1101] 2. Use generative AI to generate digital assets based on the prompt.

[1102] 3. Place the resulting digital asset in a designated location within the metaverse.

[1103] The implementation of this system uses the following hardware and software:

[1104] Hardware: Smartphones, smart glasses, head-mounted displays

[1105] Software: OpenAI API, Metaverse API, Python

[1106] As a specific example, when a user inputs the prompt sentence "A luxurious virtual store entrance," the generative AI generates a luxurious digital asset based on that instruction. The generated digital asset is placed at the entrance of the virtual store by the server. This allows the user to generate and quickly place a suitable digital asset in a virtual store simply by inputting a prompt sentence.

[1107] The flow of the specific process in the application example 1 will be described with reference to FIG.

[1108] Step 1:

[1109] The user enters a prompt sentence at the terminal.

[1110] Specifically, the user inputs a prompt sentence indicating the content of the digital asset to be generated using a terminal such as a smartphone, smart glasses, or a head-mounted display. For example, the user inputs the prompt sentence "a luxurious virtual store entrance."

[1111] Input: Prompt text (e.g. "A luxurious virtual store entrance")

[1112] Output: The prompt is sent to the server.

[1113] Step 2:

[1114] The server receives the prompt text and sends it to the generative AI.

[1115] Specifically, the server receives a prompt sentence sent by a user and sends the prompt sentence to the generative AI, which then starts a process to generate a digital asset based on the prompt sentence.

[1116] Input: prompt statement

[1117] Output: The prompt is sent to the generative AI.

[1118] Step 3:

[1119] A generative AI generates digital assets based on the prompt text.

[1120] Specifically, the generative AI analyzes the prompt text and generates an appropriate digital asset based on its content. For example, based on the prompt text "A high-end virtual store entrance," it generates a high-end digital asset.

[1121] Input: prompt statement

[1122] Output: Generated digital assets

[1123] Step 4:

[1124] The server receives the generated digital assets and places them in specific locations within the digital world (metaverse) using the metaverse API.

[1125] Specifically, the server receives the digital assets generated by the generative AI and uses the metaverse API to place the digital assets in a specified location in the virtual store. For example, a luxurious digital asset is placed at the entrance of the virtual store.

[1126] Input: Generated digital assets, location information

[1127] Output: A digital asset is placed in a specific location in the metaverse.

[1128] Step 5:

[1129] A user uses a terminal to check a virtual store in the metaverse.

[1130] Specifically, users use devices such as smartphones, smart glasses, and head-mounted displays to check the virtual store in the metaverse and verify that the generated digital assets are correctly placed.

[1131] Input: Virtual store information in the metaverse

[1132] Output: User confirms placement of digital assets

[1133] Example 2

[1134] Next, a second embodiment of the second form will be described. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal".

[1135] In the conventional digital world (metaverse), the means by which users could generate and place new digital assets were limited, making it difficult to expand the metaverse. In addition, there was a lack of efficient methods for managing and placing the generated digital assets. This made it difficult for users to maximize their creativity, and restricted the development of the metaverse.

[1136] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1137] In this invention, the server includes a means for a user to input a prompt sentence, a means for the server to receive the prompt sentence and input it to the generative AI model, a means for the server to generate a digital asset using the generative AI model, a means for the server to store the generated digital asset in a database, a means for a terminal to obtain the digital asset from the server, a means for a user to place the digital asset in a digital world (metaverse) using the terminal, and a means for expanding the digital world (metaverse). This allows a user to efficiently generate digital assets using the generative AI model and freely place them in the metaverse.

[1138] A "User" is an individual or entity that utilizes the system to enter prompt statements and place generated digital assets within the metaverse.

[1139] A "prompt sentence" is a textual instruction sentence that a user inputs to instruct a generative AI model to generate a digital asset.

[1140] "Server" means a computer system for receiving prompt sentences, generating digital assets using a generative AI model, and storing them in a database.

[1141] A "generative AI model" is an artificial intelligence model for generating digital assets based on prompt sentences entered by a user.

[1142] "Digital assets" are digital content such as 3D models, textures, and animations that are generated by generative AI models and used within the metaverse.

[1143] "Database" means an information management system for storing generated digital assets and their metadata.

[1144] A "terminal" is hardware such as a computer or VR device that a user uses to obtain digital assets from the server and place them in the metaverse.

[1145] The "metaverse" is a virtual space where generated digital assets can be placed and where users can interact with each other.

[1146] A "means for placement" is a method or tool that allows a user to place a digital asset at a specific location in the metaverse using a terminal.

[1147] An "expansion tool" is a method or tool for increasing areas and objects within the metaverse and expanding the virtual space using generated digital assets.

[1148] The present invention is a system for generating digital assets using a generative AI model and placing them in a digital world (metaverse). A specific embodiment of the system will be described below.

[1149] First, the user inputs a prompt on the Metaverse interface. The prompt instructs the generative AI model on the content of the digital asset they want to generate. For example, they input a prompt such as "I want you to generate a 3D model of a futuristic city."

[1150] Next, the server receives the prompt sentence sent by the user. The received prompt sentence is input to the generative AI model. For example, a general artificial intelligence model (such as GPT-4 or DALL-E) can be used as the generative AI model.

[1151] The server uses a generative AI model to generate digital assets based on the prompt text. The generated digital assets can range from 3D models, textures, animations, and more. The generated digital assets are stored in a database on the server. The database also stores metadata for the generated digital assets (such as the generation date and time, the prompt text, etc.).

[1152] After that, the terminal obtains the digital asset from the server in response to the user's request. The obtained digital asset is temporarily stored in the terminal's local storage. The terminal can be the user's PC or VR device.

[1153] Users place digital assets acquired using their devices in the metaverse. For example, a 3D model of a futuristic city can be placed as a new area. In this case, users can intuitively operate the assets using a VR device or PC.

[1154] For example, if a user inputs the prompt "I want you to generate a 3D model of a futuristic city," the server uses the generative AI model to generate a 3D model of a futuristic city. This 3D model is stored on the server and sent to the user's device. The user can place this 3D model in the metaverse and use it as a new region.

[1155] Examples of prompts include the following:

[1156] "I want you to generate a 3D model of a medieval castle."

[1157] "I want you to generate a 3D model of a futuristic city."

[1158] "I want you to generate a fantasy forest texture."

[1159] This system allows users to efficiently generate digital assets using generative AI models and place them freely in the metaverse, making it easier to expand the metaverse and maximizing users' creativity.

[1160] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1161] Step 1:

[1162] The user enters a prompt sentence.

[1163] A user enters "I want you to generate a 3D model of a futuristic city" into a text box on the Metaverse interface and clicks the send button. The entered prompt is sent to the server.

[1164] Step 2:

[1165] The server receives the prompt sentence and inputs it into the generative AI model.

[1166] The server receives the prompt text sent by the user. It sends the received prompt text as a request to the API of the generative AI model. The input is the prompt text, and the output is the digital asset data generated by the generative AI model.

[1167] Step 3:

[1168] The server generates the digital assets using the generative AI model.

[1169] The server receives the digital asset data returned by the generative AI model and converts it into a 3D model file format (e.g., OBJ or FBX). The input is the data from the generative AI model, and the output is the converted 3D model file.

[1170] Step 4:

[1171] The server stores the generated digital assets in a database.

[1172] The server saves the generated 3D model to a database and logs the saving completion status. The input is the 3D model file and the output is the saving completion status to the database.

[1173] Step 5:

[1174] The terminal obtains the digital asset from the server.

[1175] The terminal sends a request to the server in response to the user's operation and downloads the generated 3D model. After the download is complete, it is saved in local storage. The input is the request to the server, and the output is the 3D model file saved in local storage.

[1176] Step 6:

[1177] A user places a digital asset in the metaverse using a terminal.

[1178] The user puts on the VR device and switches to edit mode in the metaverse. The user selects the acquired 3D model and places it in the desired position by dragging and dropping it. After placing it, the user clicks the save button to confirm the changes. The input is the 3D model file stored in the local storage, and the output is the 3D model placed in the metaverse.

[1179] (Application example 2)

[1180] Next, a description will be given of Application Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server", and the headset type terminal 314 will be referred to as a "terminal".

[1181] In conventional metaverse platforms, the process for users to generate and deploy digital assets is complicated and requires specialized knowledge, making it difficult for general users to use. In addition, in virtual stores, it is difficult to freely customize store layouts and product displays, preventing users from fully expressing their creativity.

[1182] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1183] In this invention, the server includes a means for generating digital assets using a generative AI, a means for placing the generated digital assets in a digital world (metaverse), a means for expanding the digital world (metaverse), a means for the generative AI to generate digital assets in response to a user inputting a prompt sentence, and a means for placing the generated digital assets in a virtual store. This allows a user to generate digital assets by simply inputting a prompt sentence without requiring specialized knowledge, and to freely place them in a virtual store.

[1184] "Generative AI" is artificial intelligence that automatically generates digital assets based on prompts entered by users.

[1185] "Digital assets" is a general term for virtual objects and resources used within the digital world (metaverse).

[1186] The "digital world (metaverse)" is a virtual space built on the Internet, and is a platform where users can interact and engage in activities through avatars.

[1187] A "prompt sentence" is text that a user enters to instruct the generative AI on the content of the digital asset they want to generate.

[1188] A "virtual store" is a virtual store that exists in the digital world (metaverse) where users can browse and purchase products.

[1189] A "means of placement" is a method or technology for placing a generated digital asset at a specific location within the digital world (metaverse).

[1190] "Augmentation tools" are methods and technologies for expanding the scope and content of the digital world (the metaverse).

[1191] The system for implementing this invention includes a means for generating digital assets using a generative AI, placing them in a digital world (metaverse), and further expanding the digital world. Specifically, the generative AI generates digital assets when a user inputs a prompt sentence, and the generated digital assets can be placed in a virtual store.

[1192] 1. System Programming

[1193] The system uses the following hardware and software:

[1194] Hardware: Smartphones, smart glasses, head-mounted displays

[1195] Software: OpenAI API, Unreal Engine

[1196] 2. Explanation of program processing

[1197] The server receives the prompt text entered by the user and sends it to the generative AI. The generative AI generates a digital asset based on the prompt text and returns the data to the server. The server reads the returned digital asset data using Unreal Engine and places it in the specified location.

[1198] The specific process is as follows:

[1199] 1. The user uses a smartphone or head-mounted display to input a prompt describing the digital asset they want to generate.

[1200] 2. The server sends this prompt to the OpenAI API, and the generative AI generates the digital asset data.

[1201] 3. The generated digital asset data is returned to the server, which uses the Unreal Engine to load it and place it in the specified location within the virtual store.

[1202] 3. Specific Examples

[1203] For example, if the user types "I want to create a new shelf", the prompt text might look like this:

[1204] Example prompt:

[1205] "Generate a new product shelf. The shelf is made of wood and has 5 shelves, with different types of products on each shelf."

[1206] When this prompt is sent to the generative AI, the AI ​​generates a five-tier wooden shelf with different types of products on each shelf, and returns a digital asset that the server can then place in a virtual store.

[1207] In this way, users can generate digital assets by simply entering prompt sentences without the need for specialized knowledge, and place them freely in a virtual store.

[1208] The flow of the specific process in the application example 2 will be described with reference to FIG.

[1209] Step 1:

[1210] The user uses a smartphone or a head-mounted display to input a description of the digital asset to be generated as a prompt sentence.

[1211] Input: A prompt to be entered by the user (e.g. "Create a new shelf. The shelf is made of wood and has five shelves, each with a different type of product on it.")

[1212] Output: The prompt text is sent to the server.

[1213] Step 2:

[1214] The server sends the received prompt text to the OpenAI API, and the generative AI generates data for the digital asset.

[1215] Input: The prompt received by the server

[1216] Output: Digital asset data generated by generative AI

[1217] Specific operation: The server sends the prompt text to the OpenAI API, and the generative AI generates a digital asset based on the prompt text.

[1218] Step 3:

[1219] The generated digital asset data is returned to the server.

[1220] Input: Digital asset data generated by generative AI

[1221] Output: The digital asset data is returned to the server.

[1222] Specific operation: The generative AI returns the generated digital asset data to the server.

[1223] Step 4:

[1224] The server loads the returned digital asset data using the Unreal Engine and places it in the specified location within the virtual store.

[1225] Input: The digital asset's data returned by the server

[1226] Output: Digital assets are placed in a virtual store.

[1227] Specific operation: The server uses Unreal Engine to load the digital asset data and place it in the specified location in the virtual store.

[1228] Step 5:

[1229] Users can view the digital assets placed in the virtual store through their smartphones or head-mounted displays.

[1230] Input: Digital assets placed in a virtual store

[1231] Output: A representation of the digital asset for the user to see

[1232] Specific operation: The user uses a smartphone or a head-mounted display to visually check the digital assets placed in the virtual store.

[1233] In addition, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1234] "Example 1"

[1235] In one embodiment of the present invention, a generative AI and an emotion engine are combined. The generative AI generates digital assets. Meanwhile, the emotion engine recognizes the user's emotions. Specifically, the emotion is estimated from the user's facial expressions, tone of voice, text input, and the like. The output of this emotion engine is fed back to the generative AI, and the generation of the digital asset is adjusted according to the user's emotions. For example, when the user feels joy, the generative AI generates a digital asset with a bright color tone. Conversely, when the user feels sad, the generative AI generates a digital asset with a dark color tone.

[1236] "Example 2"

[1237] In another embodiment of the present invention, a generative AI, an emotion engine, and a digital world (metaverse) are combined. The generative AI generates digital assets, and the emotion engine recognizes the user's emotions. The digital world (metaverse) then arranges the generated digital assets and adjusts the arrangement according to the user's emotions. For example, when a user feels joy, the digital world (metaverse) arranges digital assets in bright tones around the user. Conversely, when a user feels sad, the digital world (metaverse) arranges digital assets in dark tones around the user.

[1238] The process flow of each embodiment will be described below.

[1239] "Example 1"

[1240] Step 1: The emotion engine recognizes the user's emotions. Specifically, it estimates emotions from the user's facial expressions, tone of voice, text input, etc.

[1241] Step 2: The output of the emotion engine is fed back to the generative AI.

[1242] Step 3: The generative AI adjusts the generation of digital assets according to the user's emotions. For example, when the user feels happy, the generative AI generates digital assets with bright tones. Conversely, when the user feels sad, the generative AI generates digital assets with dark tones.

[1243] "Example 2"

[1244] Step 1: The emotion engine recognizes the user's emotions. Specifically, it estimates emotions from the user's facial expressions, tone of voice, text input, etc.

[1245] Step 2: Generative AI generates digital assets.

[1246] Step 3: The digital world (metaverse) populates with the generated digital assets.

[1247] Step 4: The digital world (metaverse) adjusts the placement of digital assets according to the user's emotions. For example, when the user feels happy, the digital world (metaverse) places digital assets with bright colors around the user. Conversely, when the user feels sad, the digital world (metaverse) places digital assets with dark colors around the user.

[1248] Example 1

[1249] Next, a description will be given of Example 1 of the embodiment 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal".

[1250] In conventional digital asset generation systems, it was difficult to generate digital assets according to the user's emotions, and it was not possible to increase user satisfaction. In addition, there was a lack of means to effectively place the generated digital assets in the digital world and improve the user's experience.

[1251] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1252] In this invention, the server includes a means for generating digital assets using generative AI, a means for placing the generated digital assets in the digital world, a means for recognizing a user's emotions, a means for feeding back the recognized emotion data to the generative AI and adjusting the generation of the digital assets, and a means for expanding the digital world. This enables the generation of digital assets according to the user's emotions, thereby increasing the user's satisfaction. In addition, it is possible to effectively place the generated digital assets in the digital world, improving the user's experience.

[1253] "Generative AI" is an artificial intelligence technology for automatically generating digital assets.

[1254] "Digital Asset" means an asset or object that exists in digital form, such as an image, video, audio, or 3D model.

[1255] "Digital world" refers to computer-generated virtual spaces, including virtual reality and augmented reality.

[1256] "Means for recognizing user emotions" refers to technology that estimates emotions by analyzing a user's facial expressions, tone of voice, text input, etc.

[1257] "Emotion data" is data that indicates the user's emotional state, and includes emotions such as "happiness," "sadness," and "excitement," for example.

[1258] "Feedback" is the process of returning information to the system to adjust its next actions or processing based on the information it has obtained.

[1259] "Means of expanding the digital world" are technologies that increase the content and functionality of a virtual space by adding digital assets.

[1260] This invention uses a system that combines generative AI and an emotion engine to generate digital assets according to the user's emotions and place them in the digital world. A specific embodiment of this system will be described below.

[1261] The server generates a program that combines a generative AI and an emotion engine. This program has the function of generating digital assets and adjusting the generation according to the user's emotions. For example, models such as OpenAI's DALL-E and GPT-3 can be used as generative AI. For example, the emotion recognition API of Microsoft's Azure Cognitive Services can be used as the emotion engine.

[1262] A user requests the generation of a digital asset using a device such as a smartphone or PC. For example, the user enters the text "I want to create a new avatar." The device uses the built-in camera and microphone to capture the user's facial expressions and tone of voice, which are then analyzed by the emotion engine. For example, the camera detects the user's smile, and the microphone analyzes a bright tone of voice.

[1263] The terminal sends the analysis result to the server as data that "the user is happy." Based on the received emotion data, the server sends a prompt to the generative AI saying, "The user is happy, so please generate a bright and colorful avatar." The generative AI generates a bright and colorful avatar based on the prompt. For example, using DALL-E, an avatar wearing colorful clothes is generated.

[1264] The server places the generated avatar as an avatar in the user's metaverse. For example, the server places a new avatar in the home area in the user's metaverse. The user logs into the metaverse through a terminal and checks the new avatar. The user is satisfied with the generated avatar.

[1265] As a concrete example, consider a user who wants to create a new avatar in the metaverse. The user types the text "I want to create a new avatar" into the device. The emotion engine recognizes from the user's input and facial expression that the user is excited. The server receives this information and instructs the generative AI to generate a bright and colorful avatar.

[1266] Example of a prompt to be input to a generative AI model:

[1267] "Users want to create new avatars. Users are excited. Generate bright, colorful avatars."

[1268] In this way, digital assets corresponding to the user's emotions are generated and placed in the digital world, thereby increasing the user's satisfaction.

[1269] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1270] Step 1:

[1271] A user requests the creation of a digital asset.

[1272] Users use a device such as a smartphone or PC to open a dedicated application and enter the text, "I would like to create a new avatar."

[1273] Input: User text input (e.g. "I would like to create a new avatar")

[1274] Output: User request data

[1275] Step 2:

[1276] The terminal recognizes the user's emotions.

[1277] The device uses the built-in camera and microphone to capture the user's facial expressions and tone of voice, which are then analyzed by the emotion engine. For example, the camera can detect a user's smile, and the microphone can detect a bright tone of voice.

[1278] Input: User's facial expression data, voice tone data

[1279] Output: User emotion data (e.g. "The user is happy")

[1280] Step 3:

[1281] The terminal transmits the emotion data to the server.

[1282] The terminal transmits the analysis result to the server as data indicating that "the user is happy."

[1283] Input: User emotion data

[1284] Output: Send emotion data to the server

[1285] Step 4:

[1286] The server receives the emotion data and sends prompts to the generative AI.

[1287] Based on the emotional data it receives, the server sends a prompt to the generative AI saying, "The user is happy, so please generate a bright and colorful avatar."

[1288] Input: User emotion data

[1289] Output: A prompt for the generative AI (e.g., "The user is happy, so please generate a bright and colorful avatar.")

[1290] Step 5:

[1291] Generative AI generates digital assets.

[1292] The generative AI will generate bright and colorful avatars based on prompts, for example using DALL-E to generate an avatar wearing colorful clothing.

[1293] Input: Prompt statement

[1294] Output: Generated digital assets (e.g. bright and colorful avatars)

[1295] Step 6:

[1296] The server places the generated digital assets in the metaverse.

[1297] The server places the generated avatar as an avatar in the user's metaverse, for example by placing the new avatar in a home area in the user's metaverse.

[1298] Input: Generated digital assets

[1299] Output: Digital assets placed in the metaverse

[1300] Step 7:

[1301] The user reviews the generated digital assets.

[1302] The user logs into the metaverse through the terminal and checks the new avatar. The user is satisfied with the generated avatar.

[1303] Input: Digital assets placed within the metaverse

[1304] Output: User confirmation and satisfaction

[1305] (Application example 1)

[1306] Next, a description will be given of Application Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server", and the headset type terminal 314 will be referred to as a "terminal".

[1307] Conventional digital asset generation systems do not provide personalization according to the user's emotions, and the user experience is uniform. In addition, the shopping experience in the virtual store is uniform, and there is also the issue that it is not possible to flexibly respond to the user's emotions. This leads to a decrease in user satisfaction, and it is difficult to promote the use of virtual stores.

[1308] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1309] In this invention, the server includes a means for generating digital assets using generative AI, a means for arranging the generated digital assets in a digital world (metaverse), a means for adjusting the generation of digital assets according to the user's emotions using an emotion engine that recognizes the user's emotions, a means for exploring a virtual store using a smartphone, and a means for generating and arranging personalized digital assets based on the emotions, and a means for expanding the digital world (metaverse). This enables a personalized shopping experience according to the user's emotions.

[1310] "Generative AI" is an artificial intelligence technology for automatically generating digital assets.

[1311] "Digital assets" refers to assets or content that exist in digital form, including, for example, NFTs.

[1312] The "digital world (metaverse)" refers to a digital space that includes virtual reality and augmented reality, and is an environment that users can experience interactively.

[1313] An "emotion engine" is a technology for recognizing and estimating emotions from a user's facial expressions, tone of voice, text input, etc.

[1314] A "virtual store" is a virtual store that exists in the digital world (metaverse) where users can browse and purchase products.

[1315] A "personalized shopping experience" refers to a shopping experience that is customized according to a user's individual feelings and preferences.

[1316] A "smartphone" is a mobile phone that can connect to the Internet and run applications.

[1317] "NFT" stands for Non-Fungible Token, a token that uses blockchain technology to prove ownership of digital assets.

[1318] The system for implementing the present invention uses generative AI to generate digital assets and arranges the digital assets personalized according to the user's emotions in a virtual store. A specific embodiment of this system will be described below.

[1319] System Configuration

[1320] The system consists of the following main components:

[1321] 1. Generative AI: Artificial intelligence technology for automatically generating digital assets.

[1322] 2. Emotion engine: Technology that recognizes and estimates emotions from the user's facial expressions, tone of voice, text input, etc.

[1323] 3. Smartphone: A mobile phone that can connect to the Internet and run applications.

[1324] 4. Virtual Store: A virtual store that exists within the digital world (metaverse).

[1325] 5. Metaverse API: An API for placing digital assets within the metaverse.

[1326] Program Processing

[1327] The server captures the user's image using the smartphone camera and inputs it into the emotion engine. The emotion engine recognizes the user's emotion in real time and feeds the result back to the generative AI. The generative AI generates digital assets according to the user's emotion and places them in the virtual store using the metaverse API.

[1328] Hardware and Software Used

[1329] Hardware: Smartphone camera

[1330] software:

[1331] OpenCV: Camera image capture and processing

[1332] EmotionEngine: An emotion engine for recognizing user emotions

[1333] AssetGenerator: A generative AI that generates digital assets based on emotions

[1334] MetaverseAPI: API for placing digital assets in the metaverse

[1335] Examples

[1336] For example, while a user is exploring a virtual store using their smartphone, the camera captures the user's smile. The emotion engine recognizes this smile as "joy" and the generative AI generates bright and colorful digital assets (e.g. special promotional products or decorations). The metaverse API places this digital asset in a specific location in the virtual store, allowing the user to enjoy a shopping experience that matches their emotions.

[1337] Examples of prompt statements

[1338] "A prompt for a digital asset to generate when the user is delighted: 'Generate bright and colorful promotional merchandise.'"

[1339] In this way, a personalized shopping experience can be provided according to the user's emotions.

[1340] The flow of the specific process in the application example 1 will be described with reference to FIG.

[1341] Step 1:

[1342] The server captures the user's video using the smartphone camera. The input is the smartphone camera video and the output is the captured video data, which is used for subsequent emotion recognition processing.

[1343] Step 2:

[1344] The server inputs the captured video data into an emotion engine to recognize the user's emotion. The input is the captured video data, and the output is the recognized emotion data. The emotion engine analyzes the user's facial expressions from the video data and estimates the emotion.

[1345] Step 3:

[1346] The server feeds back the recognized emotion data to the generative AI and instructs it to generate a digital asset. The input is the recognized emotion data, and the output is the generated digital asset. The generative AI generates a prompt sentence based on the emotion data, and generates the digital asset according to the prompt sentence.

[1347] Step 4:

[1348] The server inputs the generated digital asset to the metaverse API and places it in the virtual store. The input is the generated digital asset, and the output is the digital asset placed in the virtual store. The metaverse API places the digital asset in the specified location.

[1349] Step 5:

[1350] A user uses a smartphone to explore the virtual store and check the placed digital assets. The input is the digital assets in the virtual store, and the output is the user's visual experience. The user can enjoy a personalized shopping experience according to his / her emotions.

[1351] Example 2

[1352] Next, a second embodiment of the second form will be described. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal".

[1353] In the conventional digital world (metaverse), it was difficult to dynamically generate and place digital assets according to the user's emotions. In addition, there was a lack of means to expand the digital world based on the user's emotions, so the improvement of the user experience was limited. This caused a problem of a decrease in the immersion and satisfaction that users felt in the digital world.

[1354] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1355] In this invention, the server includes a means for generating digital assets using generative AI, a means for arranging the generated digital assets in the digital world, a means for collecting user emotion data, a means for adjusting the arrangement of the digital assets based on the collected emotion data, and a means for expanding the digital world. This enables dynamic generation and arrangement of digital assets according to the user's emotions, thereby improving the user experience and expanding the digital world.

[1356] "Generative AI" is an artificial intelligence technology for automatically generating digital assets.

[1357] "Digital Asset" refers to a virtual object or environmental element used in the digital world.

[1358] "Digital world" refers to computer-generated virtual environments, including virtual reality and augmented reality.

[1359] "User emotion data" is data indicating the user's emotional state obtained from the user's facial expression, voice, etc.

[1360] An "emotion engine" is software or hardware for analyzing a user's emotion data and recognizing the user's emotional state.

[1361] The "means for adjusting the placement" refers to a function for dynamically changing the placement of digital assets based on the user's emotion data.

[1362] "Means for expanding the digital world" refers to functionality for expanding the scope and content of the digital world using generated digital assets.

[1363] DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[1364] The present invention is a system that uses generative AI to generate digital assets and arranges and adjusts them in the digital world based on the user's emotional data. A specific embodiment of the system will be described below.

[1365] Hardware and Software Used

[1366] Server: A computer system with high-performance computing power that is used to run the generative AI model. Specifically, we use OpenAI's GPT-4 as the generative AI model.

[1367] Terminal: A device equipped with a camera and microphone to collect user emotion data. Affectiva's emotion recognition software is used as the emotion engine.

[1368] Digital World (Metaverse): A computer-generated virtual environment, including virtual reality and augmented reality, that is a platform for the placement of generated digital assets.

[1369] Program Processing

[1370] 1. Digital Asset Creation

[1371] The server generates digital assets using a generative AI model. For example, the server inputs a prompt sentence such as "Generate a new fantasy castle" to the generative AI and generates a digital asset of a castle.

[1372] 2. Collecting user emotion data

[1373] The device collects the user's facial expressions and voice in real time. The device's camera captures the user's face and the microphone records the user's voice. This data is sent to the emotion engine.

[1374] 3. Emotion Data Analysis

[1375] The device uses an emotion engine to analyze the collected emotion data and recognize the user's emotional state. For example, the device analyzes the user's facial expressions and voice and determines that the user is happy.

[1376] 4. Placing and adjusting digital assets

[1377] The digital world (metaverse) arranges the generated digital assets according to the user's emotions. For example, when the user is happy, the metaverse arranges bright digital assets (e.g., bright flowers and light effects) around the user. Conversely, when the user is sad, it arranges dark digital assets (e.g., dark trees and shadow effects).

[1378] Examples

[1379] Example prompt:

[1380] "Generate a new fantasy castle"

[1381] "Generate the right digital assets when users are happy"

[1382] "Generate appropriate digital assets when users are sad"

[1383] This system collects user emotion data in real time, generates digital assets using a generative AI model based on that data, and optimally places them according to the user's emotions, thereby improving the user experience.

[1384] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1385] Step 1:

[1386] Collecting user emotion data

[1387] The device collects the user's facial expressions and voice in real time.

[1388] Input: The device's camera captures the user's face and the microphone records the user's voice.

[1389] Data processing: The collected facial expression and voice data is sent to the emotion engine, where pre-processing is performed to analyze the user's emotional state.

[1390] Output: Preprocessed emotion data is generated.

[1391] How it works: The device's camera captures the user's face and the microphone records the user's voice, and these data are sent to the emotion engine in real time.

[1392] Step 2:

[1393] Emotion data analysis

[1394] The terminal uses an emotion engine to analyze the collected emotion data and recognize the user's emotional state.

[1395] Input: Preprocessed emotion data.

[1396] Data calculation: The emotion engine analyzes facial expression data and voice data to recognize the user's emotional state (e.g., happiness, sadness, surprise, etc.).

[1397] Output: Data indicating the user's emotional state.

[1398] Specific operation: The emotion engine analyzes the user's facial expressions and voice and determines that the user is happy.

[1399] Step 3:

[1400] Creating digital assets

[1401] The server generates digital assets using a generative AI model.

[1402] Input: Data describing the user's emotional state and a prompt.

[1403] Data calculation: The server sends a prompt to the generative AI model and generates a digital asset according to the user's emotional state.

[1404] Output: The generated digital assets.

[1405] Specific operation: The server sends a prompt to the generative AI model saying, "Generate a digital asset that is appropriate when the user is happy," and generates a digital asset with a bright color tone.

[1406] Step 4:

[1407] Placing and adjusting digital assets

[1408] The digital world (metaverse) will place the generated digital assets according to the emotions of users.

[1409] Input: The generated digital asset and data indicating the user's emotional state.

[1410] Data Computation: The digital world dynamically adjusts the placement of digital assets based on the user's emotional state.

[1411] Output: Placement of digital assets according to user emotions.

[1412] Specific behavior: When the user is happy, the metaverse will place generated digital assets in light tones (e.g., bright flowers and light effects) around the user. Conversely, when the user is sad, the metaverse will place dark digital assets (e.g., dark trees and shadow effects).

[1413] Step 5:

[1414] User feedback

[1415] The terminal displays the arranged digital assets to the user and again collects the user's reaction.

[1416] Input: The placed digital asset.

[1417] Data processing: User responses are collected and pre-processed for use in the next cycle.

[1418] Output: Preprocessed user response data.

[1419] Specific operation: The generated digital asset is displayed on the device display, and the user looks at it and expresses emotion again. This emotion data is collected again and used for the next cycle.

[1420] (Application example 2)

[1421] Next, a description will be given of Application Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server", and the headset type terminal 314 will be referred to as a "terminal".

[1422] In conventional metaverse systems, it was difficult to dynamically change the environment according to the user's emotions, and the improvement of the user experience was limited. In addition, in virtual stores, product suggestions and adjustments to the store's atmosphere based on the user's emotions were not made, making it difficult to provide an optimal shopping experience for each individual user. This resulted in problems such as a decrease in user satisfaction and engagement.

[1423] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1424] In this invention, the server includes a means for generating digital assets using generative AI, a means for arranging the generated digital assets in a digital world (metaverse), a means for expanding the digital world (metaverse), a means for adjusting the arrangement of the digital assets according to the user's emotions using an emotion engine that recognizes the user's emotions, and a means for dynamically changing the atmosphere and product suggestions in a virtual store using an application installed on a smartphone. This makes it possible to dynamically change the environment according to the user's emotions, and to provide each user with an optimal shopping experience.

[1425] "Generative AI" is an artificial intelligence technology that automatically generates digital assets.

[1426] A "digital asset" is a virtual object or resource used within the digital world (metaverse).

[1427] The "digital world (metaverse)" is a virtual space or environment constructed on the Internet.

[1428] An "emotion engine" is a technology for recognizing emotions from a user's facial expressions, voice, etc.

[1429] A "smartphone" is a portable information terminal that can run a variety of applications in addition to the functions of a mobile phone.

[1430] An "application" is a software program designed to provide a particular function or service.

[1431] A "virtual store" is a virtual shopping space that exists on the Internet.

[1432] "Atmosphere" is the overall feeling or impression of a particular environment or space.

[1433] "Product suggestion" is the act of recommending a particular product to a user.

[1434] "Dynamic change" means changing in real time according to the situation or conditions.

[1435] The system for implementing the present invention includes a generative AI, an emotion engine, an application installed on a smartphone, and a metaverse platform. A detailed description of each component is provided below.

[1436] Generative AI

[1437] Generative AI is an artificial intelligence technology that automatically generates digital assets. The server uses generative AI to generate digital assets according to the user's emotions. For example, if the user is happy, the server generates digital assets in bright tones, and if the user is sad, the server generates digital assets in dark tones.

[1438] Emotion Engine

[1439] The emotion engine is a technology for recognizing emotions from a user's facial expressions, voice, etc. The smartphone's camera and microphone are used to collect the user's emotion data, which the emotion engine analyzes to recognize the user's emotion.

[1440] Smartphone Application

[1441] The smartphone application dynamically changes the store layout and products using digital assets generated by generative AI while the user is shopping in the virtual store. The application receives emotion data from the emotion engine and adjusts the store's atmosphere and product suggestions based on that data.

[1442] Metaverse Platform

[1443] The metaverse platform is a platform for placing generated digital assets and expanding the digital world (metaverse). The server places the digital assets generated by generative AI on the metaverse platform and adjusts the placement according to the user's emotions.

[1444] Examples

[1445] When a user visits a virtual store and smiles through the camera, the emotion engine recognizes it as "happy." This causes the generative AI to generate bright-toned digital assets and place them on the metaverse platform. Users can enjoy shopping in a bright and fun atmosphere.

[1446] Examples of prompt statements

[1447] "When the user smiles, generate a brightly colored digital asset and place it in the virtual store."

[1448] In this way, it becomes possible to dynamically change the environment in response to the user's emotions, and to provide each individual user with an optimal shopping experience.

[1449] The flow of the specific process in the application example 2 will be described with reference to FIG.

[1450] Step 1:

[1451] The user provides emotion data via the smartphone's camera and microphone.

[1452] Input: User's facial expressions and voice

[1453] Output: Emotion data (image data and audio data)

[1454] Specific actions: A user launches a virtual store application, smiles at the camera, and speaks.

[1455] Step 2:

[1456] The terminal uses an emotion engine to analyze the emotion data and recognize the user's emotion.

[1457] Input: Emotion data (image data or audio data)

[1458] Output: User's emotion (e.g. happy, sad)

[1459] Specific operation: The smartphone application sends data obtained from the camera and microphone to the emotion engine, which then analyzes the data and recognizes the user's emotions.

[1460] Step 3:

[1461] The server uses generative AI to generate digital assets based on the recognized emotions.

[1462] Input: User emotion (e.g. happy, sad)

[1463] Output: Digital Assets (e.g. light objects, dark objects)

[1464] Specific operation: Based on the emotion data received by the server from the emotion engine, the generative AI is activated and a digital asset with an appropriate theme is generated.

[1465] Step 4:

[1466] The server places the generated digital assets on the metaverse platform.

[1467] Input: Digital assets (e.g. light-toned objects, dark-toned objects)

[1468] Output: Placed digital assets

[1469] Specific operation: The server sends the digital assets received from the generative AI to the metaverse platform and places them in the specified location.

[1470] Step 5:

[1471] The terminal receives data from the metaverse platform and dynamically changes the atmosphere and product suggestions within the virtual store.

[1472] Input: Placed digital assets

[1473] Output: Modified virtual store layout and product suggestions

[1474] How it works: A smartphone application receives data from the metaverse platform and updates the user interface to show the user new moods and product suggestions.

[1475] In this way, it becomes possible to dynamically change the environment in response to the user's emotions, and to provide each individual user with an optimal shopping experience.

[1476] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires voice indicating a user input for the result of the specific processing. The control unit 46A transmits voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[1477] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ) and other generation AIs. The data generation model 58 is obtained by making a neural network perform deep learning. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating a voice, text data indicating a text, and image data indicating an image is input. The data generation model 58 performs inference on the input inference data according to the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1478] Another example of generative AI is Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) are mentioned.

[1479] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[1480] [Fourth embodiment]

[1481] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[1482] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1483] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a wide area network (WAN) and / or a local area network (LAN).

[1484] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. In addition, the microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[1485] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs the voice according to instructions from the processor 46.

[1486] Camera 42 is a small digital camera equipped with an optical system including a lens, an aperture, and a shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (e.g., an imaging range defined by an angle of view equivalent to the width of the field of vision of an average healthy person).

[1487] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for transmitting and receiving various types of information between the processor 46 and the processor 28 via the network 54. The transmission and reception of various types of information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is performed in a secure state.

[1488] The control target 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, legs, etc. The posture and behavior of the robot 414 are controlled by controlling the motors of the arms, hands, legs, etc. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[1489] Fig. 8 shows an example of main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1490] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32, and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1491] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1492] In the robot 414, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50, and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1493] Next, the specific processing by the specific processing unit 290 of the data processing device 12 will be described.

[1494] "Example 1"

[1495] The present invention is a system for generating digital assets using a generative AI. Specifically, the generative AI mass-produces digital assets such as NFTs. The generated digital assets are placed in the digital world (metaverse). For example, the digital assets generated by the generative AI are placed as specific locations or objects in the metaverse.

[1496] "Example 2"

[1497] Furthermore, an embodiment of the present invention is a system for expanding the digital world (metaverse). Specifically, the digital world (metaverse) functions as a platform that can fully utilize digital assets generated by generative AI. For example, the metaverse creates and expands new regions, buildings, and objects using digital assets generated by generative AI.

[1498] The process flow of each embodiment will be described below.

[1499] "Example 1"

[1500] Step 1: Generative AI generates digital assets such as NFTs. This generation process is based on specific algorithms and rules.

[1501] Step 2: The generated digital assets are placed in the digital world (metaverse). Placement is done as a specific place or object. For example, the digital assets are placed as part of a building or scenery in the metaverse.

[1502] "Example 2"

[1503] Step 1: The digital world (metaverse) will function as a platform where digital assets generated by generative AI can be fully utilized.

[1504] Step 2: The metaverse creates and expands new regions, buildings, and objects using digital assets generated by the generative AI, for example to develop new regions and improve existing ones.

[1505] Example 1

[1506] Next, a description will be given of Example 1 of the embodiment 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal".

[1507] Conventional digital asset generation systems have a problem in that the process for users to generate specific digital assets is complicated, and it is time-consuming to place the generated digital assets in the metaverse. In addition, there is a lack of a platform for efficiently mass-producing the generated digital assets and utilizing them in the metaverse.

[1508] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1509] In this invention, the server includes a means for a user to input a prompt sentence, a means for a terminal to send a prompt sentence to the server, a means for the server to analyze the prompt sentence, a means for the server to input a prompt sentence to a generative AI model, a means for the generative AI model to generate a digital asset, a means for the server to receive the generated digital asset, a means for the server to place the digital asset in a digital world (metaverse), and a means for the user to confirm the generated digital asset in the digital world (metaverse). This allows the user to simply input a prompt sentence to generate various digital assets using the generative AI model and place them in the metaverse.

[1510] "User" means an individual or entity that uses the system to generate and verify Digital Assets within the Metaverse.

[1511] A "prompt sentence" is text that describes the specifications and characteristics of the digital asset that the user wants to generate.

[1512] A "terminal" is an electronic device that a user uses to enter prompt text and send it to a server.

[1513] A "server" is a computer system that parses prompt statements, inputs them into the generative AI model, receives the generated digital assets, and places them in the metaverse.

[1514] A “generative AI model” is an artificial intelligence model that generates digital assets based on a prompt sentence.

[1515] "Digital Asset" means an asset in digital form, such as image data or a 3D model, that is generated by a generative AI model.

[1516] The "metaverse" is a virtual space in which digital assets are located and which users can access and use.

[1517] DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[1518] The present invention is a system for generating digital assets using a generative AI model and placing the digital assets in the metaverse. A specific embodiment of the system will be described below.

[1519] 1. System Configuration

[1520] The system consists of a terminal where the user inputs a prompt sentence, a server that analyzes the prompt sentence and inputs it into the generative AI model, and infrastructure for placing the generated digital assets in the metaverse.

[1521] 2. Hardware and Software Used

[1522] Terminal: A device through which a user enters a prompt, such as a computer, smartphone, or tablet.

[1523] Server: A computer system for parsing prompts and inputting them into a generative AI model. The server parses the prompts using a natural language processing (NLP) library (e.g., spaCy or NLTK).

[1524] Generative AI model: An artificial intelligence model that generates digital assets based on a prompt, such as OpenAI's GPT-4 or DALL-E.

[1525] Metaverse: A virtual space where generated digital assets are placed. Digital assets are placed using the metaverse API.

[1526] 3. Data processing and calculation

[1527] Prompt analysis: The server analyzes the prompt entered by the user using natural language processing technology. For example, the server extracts the keyword "blue dragon" from the prompt "Please generate an NFT of a blue dragon."

[1528] Digital asset generation: The generative AI model generates digital assets based on the parsed prompt sentences. The generated digital assets are output as image data or 3D models.

[1529] Placing digital assets: The server places the generated digital assets in a specific location in the metaverse. Using the metaverse API, the server places the digital assets in the virtual space.

[1530] 4. Specific Examples

[1531] The user inputs a prompt sentence saying "I want you to generate a red phoenix NFT." The terminal sends this prompt sentence to the server. The server parses the prompt sentence and extracts the keyword "red phoenix." The server then inputs this keyword into the generative AI model and generates a red phoenix digital asset. The generated digital asset is placed in a specific area in the metaverse by the server. The user can access the metaverse and check the generated red phoenix NFT.

[1532] In this way, users can simply enter a prompt to have a generative AI model generate various digital assets and place them in the metaverse.

[1533] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1534] Step 1:

[1535] The user enters a prompt sentence.

[1536] Specifically, the user enters a prompt such as "Please generate an NFT of a blue dragon" into the device's input form.

[1537] Input: Prompt text (e.g. "I want you to generate an NFT of a blue dragon")

[1538] Output: The prompt text entered in the terminal

[1539] Step 2:

[1540] The terminal sends a prompt to the server.

[1541] In concrete terms, when the user clicks the send button, the terminal sends a prompt text to the server using an HTTP request.

[1542] Input: The prompt text entered into the terminal

[1543] Output: The prompt text sent to the server.

[1544] Step 3:

[1545] The server parses the prompt.

[1546] Specifically, the server uses a natural language processing (NLP) library (e.g., spaCy or NLTK) to parse the prompt and extract the keyword "blue dragon."

[1547] Input: The prompt text sent to the server

[1548] Output: Parsed keywords (e.g. "blue dragon")

[1549] Step 4:

[1550] The server inputs a prompt sentence into the generative AI model.

[1551] Specifically, the server sends an API request to input the analyzed keywords into a generative AI model (e.g., GPT-4 or DALL-E).

[1552] Input: Parsed keyword

[1553] Output: The request sent to the generative AI model

[1554] Step 5:

[1555] A generative AI model generates digital assets.

[1556] Specifically, the generative AI model runs an image generation algorithm internally to generate the "blue dragon" digital asset.

[1557] Input: The request sent to the generative AI model

[1558] Output: Generated digital assets (e.g. image data of a blue dragon)

[1559] Step 6:

[1560] The server receives the generated digital asset.

[1561] Specifically, the server receives the digital assets returned from the generative AI model and stores them in a database.

[1562] Input: Generated digital assets

[1563] Output: Digital assets stored in a database

[1564] Step 7:

[1565] The server places the digital assets in the metaverse.

[1566] Specifically, the server uses the metaverse's API to place the generated digital assets in specific locations in the virtual space.

[1567] Input: Digital assets stored in a database

[1568] Output: Digital assets placed in the metaverse

[1569] Step 8:

[1570] The user views the generated digital assets in the metaverse.

[1571] Specifically, users will access the metaverse using a VR headset or PC and view the placed blue dragon NFT.

[1572] Input: Digital assets placed within the metaverse

[1573] Output: Digital assets viewed by the user

[1574] (Application example 1)

[1575] Next, a description will be given of Application Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal".

[1576] In conventional digital asset generation systems, it was difficult for users to easily generate and place digital assets suitable for specific locations in a virtual store. In addition, there was a lack of means to effectively utilize the generated digital assets in the metaverse. This resulted in a problem that virtual store owners could not quickly create and place their own digital assets, making store operations in the metaverse inefficient.

[1577] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1578] In this invention, the server includes a means for generating digital assets using generative AI, a means for arranging the generated digital assets in a digital world (metaverse), a means for expanding the digital world (metaverse), a means for generating digital assets based on a prompt sentence input by a user, and a means for arranging the generated digital assets in a specific location in a virtual store. This makes it possible for a user to generate and quickly arrange suitable digital assets in a virtual store simply by inputting a prompt sentence.

[1579] "Generative AI" is an artificial intelligence system that automatically generates digital assets based on prompts entered by users.

[1580] A "digital asset" is any digital form of property used within the digital world (metaverse), including, for example, NFTs.

[1581] The "digital world (metaverse)" is a virtual space built on the Internet, and is a platform where users can utilize digital assets to take action.

[1582] A "prompt" is text that a user inputs into a generative AI that dictates the content of the digital asset to be generated.

[1583] A "virtual store" is a virtual store that exists in the digital world (metaverse) and is a place where users can purchase goods and services.

[1584] A "means of placement" is a technical means for placing a generated digital asset at a specific location within the digital world (metaverse).

[1585] The following system configuration will be described as an embodiment of the present invention.

[1586] First, the server generates a digital asset using a generative AI. A generative AI is an artificial intelligence system that automatically generates a digital asset based on a prompt sentence entered by a user. A user enters a prompt sentence using a device such as a smartphone, smart glasses, or a head-mounted display. For example, by entering the prompt sentence "a luxurious virtual store entrance," the generative AI generates a digital asset based on that instruction.

[1587] The generated digital assets are placed in the digital world (metaverse) by the server. The digital world (metaverse) is a virtual space built on the Internet and is a platform where users can use digital assets to take part in activities. The server provides the technical means to place the generated digital assets in specific locations within the virtual store.

[1588] Specifically, the server does the following:

[1589] 1. Receive the prompt text entered by the user.

[1590] 2. Use generative AI to generate digital assets based on the prompt.

[1591] 3. Place the resulting digital asset in a designated location within the metaverse.

[1592] The implementation of this system uses the following hardware and software:

[1593] Hardware: Smartphones, smart glasses, head-mounted displays

[1594] Software: OpenAI API, Metaverse API, Python

[1595] As a specific example, when a user inputs the prompt sentence "A luxurious virtual store entrance," the generative AI generates a luxurious digital asset based on that instruction. The generated digital asset is placed at the entrance of the virtual store by the server. This allows the user to generate and quickly place a suitable digital asset in a virtual store simply by inputting a prompt sentence.

[1596] The flow of the specific process in the application example 1 will be described with reference to FIG.

[1597] Step 1:

[1598] The user enters a prompt sentence at the terminal.

[1599] Specifically, the user inputs a prompt sentence indicating the content of the digital asset to be generated using a terminal such as a smartphone, smart glasses, or a head-mounted display. For example, the user inputs the prompt sentence "a high-end virtual store entrance."

[1600] Input: Prompt text (e.g. "A luxurious virtual store entrance")

[1601] Output: The prompt is sent to the server.

[1602] Step 2:

[1603] The server receives the prompt text and sends it to the generative AI.

[1604] Specifically, the server receives a prompt sentence sent by a user and sends the prompt sentence to the generative AI, which then starts a process to generate a digital asset based on the prompt sentence.

[1605] Input: prompt statement

[1606] Output: The prompt is sent to the generative AI.

[1607] Step 3:

[1608] A generative AI generates digital assets based on the prompt text.

[1609] Specifically, the generative AI analyzes the prompt text and generates an appropriate digital asset based on its content. For example, based on the prompt text "A high-end virtual store entrance," it generates a high-end digital asset.

[1610] Input: prompt statement

[1611] Output: Generated digital assets

[1612] Step 4:

[1613] The server receives the generated digital assets and places them in specific locations within the digital world (metaverse) using the metaverse API.

[1614] Specifically, the server receives the digital assets generated by the generative AI and uses the metaverse API to place the digital assets in a specified location in the virtual store. For example, a luxurious digital asset is placed at the entrance of the virtual store.

[1615] Input: Generated digital assets, location information

[1616] Output: A digital asset is placed in a specific location in the metaverse.

[1617] Step 5:

[1618] A user uses a terminal to check a virtual store in the metaverse.

[1619] Specifically, users use devices such as smartphones, smart glasses, and head-mounted displays to check the virtual store in the metaverse and verify that the generated digital assets are correctly placed.

[1620] Input: Virtual store information in the metaverse

[1621] Output: User confirms placement of digital assets

[1622] Example 2

[1623] Next, a second embodiment of the second embodiment will be described. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal".

[1624] In the conventional digital world (metaverse), the means by which users could generate and place new digital assets were limited, making it difficult to expand the metaverse. In addition, there was a lack of efficient methods for managing and placing the generated digital assets. This made it difficult for users to maximize their creativity, and restricted the development of the metaverse.

[1625] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1626] In this invention, the server includes a means for a user to input a prompt sentence, a means for the server to receive the prompt sentence and input it to the generative AI model, a means for the server to generate a digital asset using the generative AI model, a means for the server to store the generated digital asset in a database, a means for a terminal to obtain the digital asset from the server, a means for a user to place the digital asset in a digital world (metaverse) using the terminal, and a means for expanding the digital world (metaverse). This allows a user to efficiently generate digital assets using the generative AI model and freely place them in the metaverse.

[1627] A "User" is an individual or entity that utilizes the system to enter prompt statements and place generated digital assets within the metaverse.

[1628] A "prompt sentence" is a textual instruction sentence that a user inputs to instruct a generative AI model to generate a digital asset.

[1629] "Server" means a computer system for receiving prompt sentences, generating digital assets using a generative AI model, and storing them in a database.

[1630] A "generative AI model" is an artificial intelligence model for generating digital assets based on prompt sentences entered by a user.

[1631] "Digital assets" are digital content such as 3D models, textures, and animations that are generated by generative AI models and used within the metaverse.

[1632] "Database" means an information management system for storing generated digital assets and their metadata.

[1633] A "terminal" is hardware such as a computer or VR device that a user uses to obtain digital assets from the server and place them in the metaverse.

[1634] The "metaverse" is a virtual space where generated digital assets can be placed and where users can interact with each other.

[1635] A "means for placement" is a method or tool that allows a user to place a digital asset at a specific location in the metaverse using a terminal.

[1636] An "expansion tool" is a method or tool for increasing areas and objects within the metaverse and expanding the virtual space using generated digital assets.

[1637] The present invention is a system for generating digital assets using a generative AI model and placing them in a digital world (metaverse). A specific embodiment of the system will be described below.

[1638] First, the user inputs a prompt on the Metaverse interface. The prompt instructs the generative AI model on the content of the digital asset they want to generate. For example, they input a prompt such as "I want you to generate a 3D model of a futuristic city."

[1639] Next, the server receives the prompt sentence sent by the user. The received prompt sentence is input to the generative AI model. For example, a general artificial intelligence model (such as GPT-4 or DALL-E) can be used as the generative AI model.

[1640] The server uses a generative AI model to generate digital assets based on the prompt text. The generated digital assets can range from 3D models, textures, animations, and more. The generated digital assets are stored in a database on the server. The database also stores metadata for the generated digital assets (such as the generation date and time, the prompt text, etc.).

[1641] After that, the terminal obtains the digital asset from the server in response to the user's request. The obtained digital asset is temporarily stored in the terminal's local storage. The terminal can be the user's PC or VR device.

[1642] Users place digital assets acquired using their devices in the metaverse. For example, a 3D model of a futuristic city can be placed as a new area. In this case, users can intuitively operate the assets using a VR device or PC.

[1643] For example, if a user inputs the prompt "I want you to generate a 3D model of a futuristic city," the server uses the generative AI model to generate a 3D model of a futuristic city. This 3D model is stored on the server and sent to the user's device. The user can place this 3D model in the metaverse and use it as a new region.

[1644] Examples of prompts include the following:

[1645] "I want you to generate a 3D model of a medieval castle."

[1646] "I want you to generate a 3D model of a futuristic city."

[1647] "I want you to generate a fantasy forest texture."

[1648] This system allows users to efficiently generate digital assets using generative AI models and place them freely in the metaverse, making it easier to expand the metaverse and maximizing users' creativity.

[1649] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1650] Step 1:

[1651] The user enters a prompt sentence.

[1652] A user enters "I want you to generate a 3D model of a futuristic city" into a text box on the Metaverse interface and clicks the send button. The entered prompt is sent to the server.

[1653] Step 2:

[1654] The server receives the prompt sentence and inputs it into the generative AI model.

[1655] The server receives the prompt text sent by the user. It sends the received prompt text as a request to the API of the generative AI model. The input is the prompt text, and the output is the digital asset data generated by the generative AI model.

[1656] Step 3:

[1657] The server generates the digital assets using the generative AI model.

[1658] The server receives the digital asset data returned by the generative AI model and converts it into a 3D model file format (e.g., OBJ or FBX). The input is the data from the generative AI model, and the output is the converted 3D model file.

[1659] Step 4:

[1660] The server stores the generated digital assets in a database.

[1661] The server saves the generated 3D model to a database and logs the saving completion status. The input is the 3D model file and the output is the saving completion status to the database.

[1662] Step 5:

[1663] The terminal obtains the digital asset from the server.

[1664] The terminal sends a request to the server in response to the user's operation and downloads the generated 3D model. After the download is complete, it is saved in local storage. The input is the request to the server, and the output is the 3D model file saved in local storage.

[1665] Step 6:

[1666] A user places a digital asset in the metaverse using a terminal.

[1667] The user puts on the VR device and switches to edit mode in the metaverse. The user selects the acquired 3D model and places it in the desired position by dragging and dropping it. After placing it, the user clicks the save button to confirm the changes. The input is the 3D model file stored in the local storage, and the output is the 3D model placed in the metaverse.

[1668] (Application example 2)

[1669] Next, a description will be given of Application Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal".

[1670] In conventional metaverse platforms, the process for users to generate and deploy digital assets is complicated and requires specialized knowledge, making it difficult for general users to use. In addition, in virtual stores, it is difficult to freely customize store layouts and product displays, preventing users from fully expressing their creativity.

[1671] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1672] In this invention, the server includes a means for generating digital assets using a generative AI, a means for placing the generated digital assets in a digital world (metaverse), a means for expanding the digital world (metaverse), a means for the generative AI to generate digital assets in response to a user inputting a prompt sentence, and a means for placing the generated digital assets in a virtual store. This allows a user to generate digital assets by simply inputting a prompt sentence without requiring specialized knowledge, and to freely place them in a virtual store.

[1673] "Generative AI" is artificial intelligence that automatically generates digital assets based on prompts entered by users.

[1674] "Digital assets" is a general term for virtual objects and resources used within the digital world (metaverse).

[1675] The "digital world (metaverse)" is a virtual space built on the Internet, and is a platform where users can interact and engage in activities through avatars.

[1676] A "prompt sentence" is text that a user enters to instruct the generative AI on the content of the digital asset they want to generate.

[1677] A "virtual store" is a virtual store that exists in the digital world (metaverse) where users can browse and purchase products.

[1678] A "means of placement" is a method or technology for placing a generated digital asset at a specific location within the digital world (metaverse).

[1679] "Augmentation tools" are methods and technologies for expanding the scope and content of the digital world (the metaverse).

[1680] The system for implementing this invention includes a means for generating digital assets using a generative AI, placing them in a digital world (metaverse), and further expanding the digital world. Specifically, the generative AI generates digital assets when a user inputs a prompt sentence, and the generated digital assets can be placed in a virtual store.

[1681] 1. System Programming

[1682] The system uses the following hardware and software:

[1683] Hardware: Smartphones, smart glasses, head-mounted displays

[1684] Software: OpenAI API, Unreal Engine

[1685] 2. Explanation of program processing

[1686] The server receives the prompt text entered by the user and sends it to the generative AI. The generative AI generates a digital asset based on the prompt text and returns the data to the server. The server reads the returned digital asset data using Unreal Engine and places it in the specified location.

[1687] The specific process is as follows:

[1688] 1. The user uses a smartphone or head-mounted display to input a prompt describing the digital asset they want to generate.

[1689] 2. The server sends this prompt to the OpenAI API, and the generative AI generates the digital asset data.

[1690] 3. The generated digital asset data is returned to the server, which uses the Unreal Engine to load it and place it in the specified location within the virtual store.

[1691] 3. Specific Examples

[1692] For example, if the user types "I want to create a new shelf", the prompt text might look like this:

[1693] Example prompt:

[1694] "Generate a new product shelf. The shelf is made of wood and has 5 shelves, with different types of products on each shelf."

[1695] When this prompt is sent to the generative AI, the AI ​​generates a five-tier wooden shelf with different types of products on each shelf, and returns a digital asset that the server can then place in a virtual store.

[1696] In this way, users can generate digital assets by simply entering prompt sentences without the need for specialized knowledge, and place them freely in a virtual store.

[1697] The flow of the specific process in the application example 2 will be described with reference to FIG.

[1698] Step 1:

[1699] The user uses a smartphone or a head-mounted display to input a description of the digital asset to be generated as a prompt sentence.

[1700] Input: A prompt to be entered by the user (e.g. "Create a new shelf. The shelf is made of wood and has five shelves, each with a different type of product on it.")

[1701] Output: The prompt text is sent to the server.

[1702] Step 2:

[1703] The server sends the received prompt text to the OpenAI API, and the generative AI generates data for the digital asset.

[1704] Input: The prompt received by the server

[1705] Output: Digital asset data generated by generative AI

[1706] Specific operation: The server sends the prompt text to the OpenAI API, and the generative AI generates a digital asset based on the prompt text.

[1707] Step 3:

[1708] The generated digital asset data is returned to the server.

[1709] Input: Digital asset data generated by generative AI

[1710] Output: The digital asset data is returned to the server.

[1711] Specific operation: The generative AI returns the generated digital asset data to the server.

[1712] Step 4:

[1713] The server loads the returned digital asset data using the Unreal Engine and places it in the specified location within the virtual store.

[1714] Input: The digital asset's data returned by the server

[1715] Output: Digital assets are placed in a virtual store.

[1716] Specific operation: The server uses Unreal Engine to load the digital asset data and place it in the specified location in the virtual store.

[1717] Step 5:

[1718] Users can view the digital assets placed in the virtual store through their smartphones or head-mounted displays.

[1719] Input: Digital assets placed in a virtual store

[1720] Output: A representation of the digital asset for the user to see

[1721] Specific operation: The user uses a smartphone or a head-mounted display to visually check the digital assets placed in the virtual store.

[1722] In addition, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1723] "Example 1"

[1724] In one embodiment of the present invention, a generative AI and an emotion engine are combined. The generative AI generates digital assets. Meanwhile, the emotion engine recognizes the user's emotions. Specifically, the emotion is estimated from the user's facial expressions, tone of voice, text input, and the like. The output of this emotion engine is fed back to the generative AI, and the generation of the digital asset is adjusted according to the user's emotions. For example, when the user feels joy, the generative AI generates a digital asset with a bright color tone. Conversely, when the user feels sad, the generative AI generates a digital asset with a dark color tone.

[1725] "Example 2"

[1726] In another embodiment of the present invention, a generative AI, an emotion engine, and a digital world (metaverse) are combined. The generative AI generates digital assets, and the emotion engine recognizes the user's emotions. The digital world (metaverse) then arranges the generated digital assets and adjusts the arrangement according to the user's emotions. For example, when a user feels joy, the digital world (metaverse) arranges digital assets in bright tones around the user. Conversely, when a user feels sad, the digital world (metaverse) arranges digital assets in dark tones around the user.

[1727] The process flow of each embodiment will be described below.

[1728] "Example 1"

[1729] Step 1: The emotion engine recognizes the user's emotions. Specifically, it estimates emotions from the user's facial expressions, tone of voice, text input, etc.

[1730] Step 2: The output of the emotion engine is fed back to the generative AI.

[1731] Step 3: The generative AI adjusts the generation of digital assets according to the user's emotions. For example, when the user feels happy, the generative AI generates digital assets with bright tones. Conversely, when the user feels sad, the generative AI generates digital assets with dark tones.

[1732] "Example 2"

[1733] Step 1: The emotion engine recognizes the user's emotions. Specifically, it estimates emotions from the user's facial expressions, tone of voice, text input, etc.

[1734] Step 2: Generative AI generates digital assets.

[1735] Step 3: The digital world (metaverse) populates with the generated digital assets.

[1736] Step 4: The digital world (metaverse) adjusts the placement of digital assets according to the user's emotions. For example, when the user feels happy, the digital world (metaverse) places digital assets with bright colors around the user. Conversely, when the user feels sad, the digital world (metaverse) places digital assets with dark colors around the user.

[1737] Example 1

[1738] Next, a description will be given of Example 1 of the embodiment 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal".

[1739] In conventional digital asset generation systems, it was difficult to generate digital assets according to the user's emotions, and it was not possible to increase user satisfaction. In addition, there was a lack of means to effectively place the generated digital assets in the digital world and improve the user's experience.

[1740] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1741] In this invention, the server includes a means for generating digital assets using generative AI, a means for placing the generated digital assets in the digital world, a means for recognizing a user's emotions, a means for feeding back the recognized emotion data to the generative AI and adjusting the generation of the digital assets, and a means for expanding the digital world. This enables the generation of digital assets according to the user's emotions, thereby increasing the user's satisfaction. In addition, it is possible to effectively place the generated digital assets in the digital world, improving the user's experience.

[1742] "Generative AI" is an artificial intelligence technology for automatically generating digital assets.

[1743] "Digital Asset" means an asset or object that exists in digital form, such as an image, video, audio, or 3D model.

[1744] "Digital world" refers to computer-generated virtual spaces, including virtual reality and augmented reality.

[1745] "Means for recognizing user emotions" refers to technology that estimates emotions by analyzing a user's facial expressions, tone of voice, text input, etc.

[1746] "Emotion data" is data that indicates the user's emotional state, and includes emotions such as "happiness," "sadness," and "excitement," for example.

[1747] "Feedback" is the process of returning information to the system to adjust its next actions or processing based on the information it has obtained.

[1748] "Means of expanding the digital world" are technologies that increase the content and functionality of a virtual space by adding digital assets.

[1749] This invention uses a system that combines generative AI and an emotion engine to generate digital assets according to the user's emotions and place them in the digital world. A specific embodiment of this system will be described below.

[1750] The server generates a program that combines a generative AI and an emotion engine. This program has the function of generating digital assets and adjusting the generation according to the user's emotions. For example, models such as OpenAI's DALL-E and GPT-3 can be used as generative AI. For example, the emotion recognition API of Microsoft's Azure Cognitive Services can be used as the emotion engine.

[1751] A user requests the generation of a digital asset using a device such as a smartphone or PC. For example, the user enters the text "I want to create a new avatar." The device uses the built-in camera and microphone to capture the user's facial expressions and tone of voice, which are then analyzed by the emotion engine. For example, the camera detects the user's smile, and the microphone analyzes a bright tone of voice.

[1752] The terminal sends the analysis result to the server as data that "the user is happy." Based on the received emotion data, the server sends a prompt to the generative AI saying, "The user is happy, so please generate a bright and colorful avatar." The generative AI generates a bright and colorful avatar based on the prompt. For example, using DALL-E, an avatar wearing colorful clothes is generated.

[1753] The server places the generated avatar as an avatar in the user's metaverse. For example, the server places a new avatar in the home area in the user's metaverse. The user logs into the metaverse through a terminal and checks the new avatar. The user is satisfied with the generated avatar.

[1754] As a concrete example, consider a user who wants to create a new avatar in the metaverse. The user types the text "I want to create a new avatar" into the device. The emotion engine recognizes from the user's input and facial expression that the user is excited. The server receives this information and instructs the generative AI to generate a bright and colorful avatar.

[1755] Example of a prompt to be input to a generative AI model:

[1756] "Users want to create new avatars. Users are excited. Generate bright, colorful avatars."

[1757] In this way, digital assets corresponding to the user's emotions are generated and placed in the digital world, thereby increasing the user's satisfaction.

[1758] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1759] Step 1:

[1760] A user requests the creation of a digital asset.

[1761] Users use a device such as a smartphone or PC to open a dedicated application and enter the text, "I would like to create a new avatar."

[1762] Input: User text input (e.g. "I would like to create a new avatar")

[1763] Output: User request data

[1764] Step 2:

[1765] The terminal recognizes the user's emotions.

[1766] The device uses the built-in camera and microphone to capture the user's facial expressions and tone of voice, which are then analyzed by the emotion engine. For example, the camera can detect a user's smile, and the microphone can detect a bright tone of voice.

[1767] Input: User's facial expression data, voice tone data

[1768] Output: User emotion data (e.g. "The user is happy")

[1769] Step 3:

[1770] The terminal transmits the emotion data to the server.

[1771] The terminal transmits the analysis result to the server as data indicating that "the user is happy."

[1772] Input: User emotion data

[1773] Output: Send emotion data to the server

[1774] Step 4:

[1775] The server receives the emotion data and sends prompts to the generative AI.

[1776] Based on the emotional data it receives, the server sends a prompt to the generative AI saying, "The user is happy, so please generate a bright and colorful avatar."

[1777] Input: User emotion data

[1778] Output: A prompt for the generative AI (e.g., "The user is happy, so please generate a bright and colorful avatar.")

[1779] Step 5:

[1780] Generative AI generates digital assets.

[1781] The generative AI will generate bright and colorful avatars based on prompts, for example using DALL-E to generate an avatar wearing colorful clothing.

[1782] Input: Prompt statement

[1783] Output: Generated digital assets (e.g. bright and colorful avatars)

[1784] Step 6:

[1785] The server places the generated digital assets in the metaverse.

[1786] The server places the generated avatar as an avatar in the user's metaverse, for example by placing the new avatar in a home area in the user's metaverse.

[1787] Input: Generated digital assets

[1788] Output: Digital assets placed in the metaverse

[1789] Step 7:

[1790] The user reviews the generated digital assets.

[1791] The user logs into the metaverse through the terminal and checks the new avatar. The user is satisfied with the generated avatar.

[1792] Input: Digital assets placed within the metaverse

[1793] Output: User confirmation and satisfaction

[1794] (Application example 1)

[1795] Next, a description will be given of Application Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal".

[1796] Conventional digital asset generation systems do not provide personalization according to the user's emotions, and the user experience is uniform. In addition, the shopping experience in the virtual store is uniform, and there is also the issue that it is not possible to flexibly respond to the user's emotions. This leads to a decrease in user satisfaction, and it is difficult to promote the use of virtual stores.

[1797] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1798] In this invention, the server includes a means for generating digital assets using generative AI, a means for arranging the generated digital assets in a digital world (metaverse), a means for adjusting the generation of digital assets according to the user's emotions using an emotion engine that recognizes the user's emotions, a means for exploring a virtual store using a smartphone, and a means for generating and arranging personalized digital assets based on the emotions, and a means for expanding the digital world (metaverse). This enables a personalized shopping experience according to the user's emotions.

[1799] "Generative AI" is an artificial intelligence technology for automatically generating digital assets.

[1800] "Digital assets" refers to assets or content that exist in digital form, including, for example, NFTs.

[1801] The "digital world (metaverse)" refers to a digital space that includes virtual reality and augmented reality, and is an environment that users can experience interactively.

[1802] An "emotion engine" is a technology for recognizing and estimating emotions from a user's facial expressions, tone of voice, text input, etc.

[1803] A "virtual store" is a virtual store that exists in the digital world (metaverse) where users can browse and purchase products.

[1804] A "personalized shopping experience" refers to a shopping experience that is customized according to a user's individual feelings and preferences.

[1805] A "smartphone" is a mobile phone that can connect to the Internet and run applications.

[1806] "NFT" stands for Non-Fungible Token, a token that uses blockchain technology to prove ownership of digital assets.

[1807] The system for implementing the present invention uses generative AI to generate digital assets and arranges the digital assets personalized according to the user's emotions in a virtual store. A specific embodiment of this system will be described below.

[1808] System Configuration

[1809] The system consists of the following main components:

[1810] 1. Generative AI: Artificial intelligence technology for automatically generating digital assets.

[1811] 2. Emotion engine: Technology that recognizes and estimates emotions from the user's facial expressions, tone of voice, text input, etc.

[1812] 3. Smartphone: A mobile phone that can connect to the Internet and run applications.

[1813] 4. Virtual Store: A virtual store that exists within the digital world (metaverse).

[1814] 5. Metaverse API: An API for placing digital assets within the metaverse.

[1815] Program Processing

[1816] The server captures the user's image using the smartphone camera and inputs it into the emotion engine. The emotion engine recognizes the user's emotion in real time and feeds the result back to the generative AI. The generative AI generates digital assets according to the user's emotion and places them in the virtual store using the metaverse API.

[1817] Hardware and Software Used

[1818] Hardware: Smartphone camera

[1819] software:

[1820] OpenCV: Camera image capture and processing

[1821] EmotionEngine: An emotion engine for recognizing user emotions

[1822] AssetGenerator: A generative AI that generates digital assets based on emotions

[1823] MetaverseAPI: API for placing digital assets in the metaverse

[1824] Examples

[1825] For example, while a user is exploring a virtual store using their smartphone, the camera captures the user's smile. The emotion engine recognizes this smile as "joy" and the generative AI generates bright and colorful digital assets (e.g. special promotional products or decorations). The metaverse API places this digital asset in a specific location in the virtual store, allowing the user to enjoy a shopping experience that matches their emotions.

[1826] Examples of prompt statements

[1827] "A prompt for a digital asset to generate when the user is delighted: 'Generate bright and colorful promotional merchandise.'"

[1828] In this way, a personalized shopping experience can be provided according to the user's emotions.

[1829] The flow of the specific process in the application example 1 will be described with reference to FIG.

[1830] Step 1:

[1831] The server captures the user's video using the smartphone camera. The input is the smartphone camera video and the output is the captured video data, which is used for subsequent emotion recognition processing.

[1832] Step 2:

[1833] The server inputs the captured video data into an emotion engine to recognize the user's emotion. The input is the captured video data, and the output is the recognized emotion data. The emotion engine analyzes the user's facial expressions from the video data and estimates the emotion.

[1834] Step 3:

[1835] The server feeds back the recognized emotion data to the generative AI and instructs it to generate a digital asset. The input is the recognized emotion data, and the output is the generated digital asset. The generative AI generates a prompt sentence based on the emotion data, and generates the digital asset according to the prompt sentence.

[1836] Step 4:

[1837] The server inputs the generated digital asset to the metaverse API and places it in the virtual store. The input is the generated digital asset, and the output is the digital asset placed in the virtual store. The metaverse API places the digital asset in the specified location.

[1838] Step 5:

[1839] A user uses a smartphone to explore the virtual store and check the placed digital assets. The input is the digital assets in the virtual store, and the output is the user's visual experience. The user can enjoy a personalized shopping experience according to his / her emotions.

[1840] Example 2

[1841] Next, a second embodiment of the second embodiment will be described. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal".

[1842] In the conventional digital world (metaverse), it was difficult to dynamically generate and place digital assets according to the user's emotions. In addition, there was a lack of means to expand the digital world based on the user's emotions, so the improvement of the user experience was limited. This caused a problem of a decrease in the immersion and satisfaction that users felt in the digital world.

[1843] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1844] In this invention, the server includes a means for generating digital assets using generative AI, a means for arranging the generated digital assets in the digital world, a means for collecting user emotion data, a means for adjusting the arrangement of the digital assets based on the collected emotion data, and a means for expanding the digital world. This enables dynamic generation and arrangement of digital assets according to the user's emotions, thereby improving the user experience and expanding the digital world.

[1845] "Generative AI" is an artificial intelligence technology for automatically generating digital assets.

[1846] "Digital Asset" refers to a virtual object or environmental element used in the digital world.

[1847] "Digital world" refers to computer-generated virtual environments, including virtual reality and augmented reality.

[1848] "User emotion data" is data indicating the user's emotional state obtained from the user's facial expression, voice, etc.

[1849] An "emotion engine" is software or hardware for analyzing a user's emotion data and recognizing the user's emotional state.

[1850] The "means for adjusting the placement" refers to a function for dynamically changing the placement of digital assets based on the user's emotion data.

[1851] "Means for expanding the digital world" refers to functionality for expanding the scope and content of the digital world using generated digital assets.

[1852] DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[1853] The present invention is a system that uses generative AI to generate digital assets and arranges and adjusts them in the digital world based on the user's emotional data. A specific embodiment of the system will be described below.

[1854] Hardware and Software Used

[1855] Server: A computer system with high-performance computing power that is used to run the generative AI model. Specifically, we use OpenAI's GPT-4 as the generative AI model.

[1856] Terminal: A device equipped with a camera and microphone to collect user emotion data. Affectiva's emotion recognition software is used as the emotion engine.

[1857] Digital World (Metaverse): A computer-generated virtual environment, including virtual reality and augmented reality, that is a platform for the placement of generated digital assets.

[1858] Program Processing

[1859] 1. Digital Asset Creation

[1860] The server generates digital assets using a generative AI model. For example, the server inputs a prompt sentence such as "Generate a new fantasy castle" to the generative AI and generates a digital asset of a castle.

[1861] 2. Collecting user emotion data

[1862] The device collects the user's facial expressions and voice in real time. The device's camera captures the user's face and the microphone records the user's voice. This data is sent to the emotion engine.

[1863] 3. Emotion Data Analysis

[1864] The device uses an emotion engine to analyze the collected emotion data and recognize the user's emotional state. For example, the device analyzes the user's facial expressions and voice and determines that the user is happy.

[1865] 4. Placing and adjusting digital assets

[1866] The digital world (metaverse) arranges the generated digital assets according to the user's emotions. For example, when the user is happy, the metaverse arranges bright digital assets (e.g., bright flowers and light effects) around the user. Conversely, when the user is sad, it arranges dark digital assets (e.g., dark trees and shadow effects).

[1867] Examples

[1868] Example prompt:

[1869] "Generate a new fantasy castle"

[1870] "Generate the right digital assets when users are happy"

[1871] "Generate appropriate digital assets when users are sad"

[1872] This system collects user emotion data in real time, generates digital assets using a generative AI model based on that data, and optimally places them according to the user's emotions, thereby improving the user experience.

[1873] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1874] Step 1:

[1875] Collecting user emotion data

[1876] The device collects the user's facial expressions and voice in real time.

[1877] Input: The device's camera captures the user's face and the microphone records the user's voice.

[1878] Data processing: The collected facial expression and voice data is sent to the emotion engine, where pre-processing is performed to analyze the user's emotional state.

[1879] Output: Preprocessed emotion data is generated.

[1880] How it works: The device's camera captures the user's face and the microphone records the user's voice, and these data are sent to the emotion engine in real time.

[1881] Step 2:

[1882] Emotion data analysis

[1883] The terminal uses an emotion engine to analyze the collected emotion data and recognize the user's emotional state.

[1884] Input: Preprocessed emotion data.

[1885] Data calculation: The emotion engine analyzes facial expression data and voice data to recognize the user's emotional state (e.g., happiness, sadness, surprise, etc.).

[1886] Output: Data indicating the user's emotional state.

[1887] Specific operation: The emotion engine analyzes the user's facial expressions and voice and determines that the user is happy.

[1888] Step 3:

[1889] Creating digital assets

[1890] The server generates digital assets using a generative AI model.

[1891] Input: Data describing the user's emotional state and a prompt.

[1892] Data calculation: The server sends a prompt sentence to the generative AI model and generates a digital asset according to the user's emotional state.

[1893] Output: The generated digital assets.

[1894] Specific operation: The server sends a prompt sentence to the generative AI model, "Generate a digital asset that is appropriate when the user is happy," and generates a digital asset with a bright color tone.

[1895] Step 4:

[1896] Placing and adjusting digital assets

[1897] The digital world (metaverse) will place the generated digital assets according to the emotions of users.

[1898] Input: The generated digital asset and data indicating the user's emotional state.

[1899] Data Computation: The digital world dynamically adjusts the placement of digital assets based on the user's emotional state.

[1900] Output: Placement of digital assets according to user emotions.

[1901] Specific behavior: When the user is happy, the metaverse will place generated digital assets in light tones (e.g., bright flowers and light effects) around the user. Conversely, when the user is sad, the metaverse will place dark digital assets (e.g., dark trees and shadow effects).

[1902] Step 5:

[1903] User feedback

[1904] The terminal displays the arranged digital assets to the user and again collects the user's reaction.

[1905] Input: The placed digital asset.

[1906] Data processing: User responses are collected and pre-processed for use in the next cycle.

[1907] Output: Preprocessed user response data.

[1908] Specific operation: The generated digital asset is displayed on the device display, and the user looks at it and expresses emotion again. This emotion data is collected again and used for the next cycle.

[1909] (Application example 2)

[1910] Next, a description will be given of Application Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal".

[1911] In conventional metaverse systems, it was difficult to dynamically change the environment according to the user's emotions, and the improvement of the user experience was limited. In addition, in virtual stores, product suggestions and adjustments to the store's atmosphere based on the user's emotions were not made, making it difficult to provide an optimal shopping experience for each individual user. This resulted in problems such as a decrease in user satisfaction and engagement.

[1912] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1913] In this invention, the server includes a means for generating digital assets using generative AI, a means for arranging the generated digital assets in a digital world (metaverse), a means for expanding the digital world (metaverse), a means for adjusting the arrangement of the digital assets according to the user's emotions using an emotion engine that recognizes the user's emotions, and a means for dynamically changing the atmosphere and product suggestions in a virtual store using an application installed on a smartphone. This makes it possible to dynamically change the environment according to the user's emotions, and to provide each user with an optimal shopping experience.

[1914] "Generative AI" is an artificial intelligence technology that automatically generates digital assets.

[1915] A "digital asset" is a virtual object or resource used within the digital world (metaverse).

[1916] The "digital world (metaverse)" is a virtual space or environment constructed on the Internet.

[1917] An "emotion engine" is a technology for recognizing emotions from a user's facial expressions, voice, etc.

[1918] A "smartphone" is a portable information terminal that can run a variety of applications in addition to the functions of a mobile phone.

[1919] An "application" is a software program designed to provide a particular function or service.

[1920] A "virtual store" is a virtual shopping space that exists on the Internet.

[1921] "Atmosphere" is the overall feeling or impression of a particular environment or space.

[1922] "Product suggestion" is the act of recommending a particular product to a user.

[1923] "Dynamic change" means changing in real time according to the situation or conditions.

[1924] The system for implementing the present invention includes a generative AI, an emotion engine, an application installed on a smartphone, and a metaverse platform. A detailed description of each component is provided below.

[1925] Generative AI

[1926] Generative AI is an artificial intelligence technology that automatically generates digital assets. The server uses generative AI to generate digital assets according to the user's emotions. For example, if the user is happy, the server generates digital assets in bright tones, and if the user is sad, the server generates digital assets in dark tones.

[1927] Emotion Engine

[1928] The emotion engine is a technology for recognizing emotions from a user's facial expressions, voice, etc. The smartphone's camera and microphone are used to collect the user's emotion data, which the emotion engine analyzes to recognize the user's emotion.

[1929] Smartphone Application

[1930] The smartphone application dynamically changes the store layout and products using digital assets generated by generative AI while the user is shopping in the virtual store. The application receives emotion data from the emotion engine and adjusts the store's atmosphere and product suggestions based on that data.

[1931] Metaverse Platform

[1932] The metaverse platform is a platform for placing generated digital assets and expanding the digital world (metaverse). The server places the digital assets generated by generative AI on the metaverse platform and adjusts the placement according to the user's emotions.

[1933] Examples

[1934] When a user visits a virtual store and smiles through the camera, the emotion engine recognizes it as "happy." This causes the generative AI to generate bright-toned digital assets and place them on the metaverse platform. Users can enjoy shopping in a bright and fun atmosphere.

[1935] Examples of prompt statements

[1936] "When the user smiles, generate a brightly colored digital asset and place it in the virtual store."

[1937] In this way, it becomes possible to dynamically change the environment in response to the user's emotions, and to provide each individual user with an optimal shopping experience.

[1938] The flow of the specific process in the application example 2 will be described with reference to FIG.

[1939] Step 1:

[1940] The user provides emotion data via the smartphone's camera and microphone.

[1941] Input: User's facial expressions and voice

[1942] Output: Emotion data (image data and audio data)

[1943] Specific actions: A user launches a virtual store application, smiles at the camera, and speaks.

[1944] Step 2:

[1945] The terminal uses an emotion engine to analyze the emotion data and recognize the user's emotion.

[1946] Input: Emotion data (image data or audio data)

[1947] Output: User emotion (e.g. happy, sad)

[1948] Specific operation: The smartphone application sends data obtained from the camera and microphone to the emotion engine, which then analyzes the data and recognizes the user's emotions.

[1949] Step 3:

[1950] The server uses generative AI to generate digital assets based on the recognized emotions.

[1951] Input: User emotion (e.g. happy, sad)

[1952] Output: Digital Assets (e.g. light objects, dark objects)

[1953] Specific operation: Based on the emotion data received by the server from the emotion engine, the generative AI is activated and a digital asset with an appropriate theme is generated.

[1954] Step 4:

[1955] The server places the generated digital assets on the metaverse platform.

[1956] Input: Digital assets (e.g. light-toned objects, dark-toned objects)

[1957] Output: Placed digital assets

[1958] Specific operation: The server sends the digital assets received from the generative AI to the metaverse platform and places them in the specified location.

[1959] Step 5:

[1960] The terminal receives data from the metaverse platform and dynamically changes the atmosphere and product suggestions within the virtual store.

[1961] Input: Placed digital assets

[1962] Output: Modified virtual store layout and product suggestions

[1963] How it works: A smartphone application receives data from the metaverse platform and updates the user interface to show the user new moods and product suggestions.

[1964] In this way, it becomes possible to dynamically change the environment in response to the user's emotions, and to provide each individual user with an optimal shopping experience.

[1965] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the control target 443 to output the result of the specific processing. The microphone 238 acquires a voice indicating a user input for the result of the specific processing. The control unit 46A transmits voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[1966] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ) and other generation AIs. The data generation model 58 is obtained by making a neural network perform deep learning. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating a voice, text data indicating a text, and image data indicating an image is input. The data generation model 58 performs inference on the input inference data according to the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1967] Another example of generative AI is Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) are mentioned.

[1968] In the above embodiment, an example was given in which the specific process was performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the robot 414.

[1969] The emotion identification model 59 as an emotion engine may determine the emotion of the user according to a specific mapping. Specifically, the emotion identification model 59 may determine the emotion of the user according to an emotion map (see FIG. 9), which is a specific mapping. Similarly, the emotion identification model 59 may determine the emotion of the robot, and the identification processing unit 290 may perform identification processing using the emotion of the robot.

[1970] FIG. 9 is a diagram showing an emotion map 400 on which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. The closer to the center of the concentric circles, the more primitive emotions are arranged. The more outside the concentric circles, the more emotions that represent states and actions that arise from a state of mind are arranged. Emotions are a concept that includes emotions and mental states. On the left side of the concentric circles, emotions that are generally generated from reactions that occur in the brain are arranged. On the right side of the concentric circles, emotions that are generally induced by situational judgment are arranged. On the upper and lower sides of the concentric circles, emotions that are generally generated from reactions that occur in the brain and are induced by situational judgment are arranged. In addition, on the upper side of the concentric circles, emotions of "pleasure" are arranged, and on the lower side, emotions of "discomfort" are arranged. In this way, in the emotion map 400, multiple emotions are mapped based on the structure in which emotions are generated, and emotions that tend to occur simultaneously are mapped close to each other.

[1971] These emotions are distributed in the 3 o'clock direction of emotion map 400 and usually fluctuate between relief and anxiety. In the right half of emotion map 400, situational awareness dominates over internal sensations, resulting in a sense of calm.

[1972] The inside of emotion map 400 represents what is going on inside one's mind, and the outside of emotion map 400 represents behavior, so the further out you go on emotion map 400, the more visible the emotions become (the more they are expressed in behavior).

[1973] Here, human emotions are based on various balances such as posture and blood sugar level, and when these balances are far from the ideal, it indicates an unpleasant state, and when they are close to the ideal, it indicates a pleasant state. Emotions can also be created for robots, cars, motorcycles, etc., based on various balances such as posture and battery level, so that when these balances are far from the ideal, it indicates an unpleasant state, and when they are close to the ideal, it indicates a pleasant state. The emotion map may be generated, for example, based on the emotion map of Dr. Mitsuyoshi (Research on speech emotion recognition and emotion brain physiological signal analysis system, Tokushima University, doctoral dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). On the left half of the emotion map, emotions belonging to an area called "reaction" where sensation is dominant are lined up. On the right half of the emotion map, emotions belonging to an area called "situation" where situation recognition is dominant are lined up.

[1974] The emotion map defines two emotions that promote learning. The first is the negative emotion around the middle of "repentance" or "remorse" on the situation side. In other words, this is when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is the positive emotion around "desire" on the response side. In other words, this is when the robot has positive feelings such as "I want more" or "I want to know more."

[1975] The emotion identification model 59 inputs the user input to a pre-trained neural network, obtains emotion values ​​indicating each emotion shown in the emotion map 400, and determines the emotion of the user. This neural network is pre-trained based on multiple learning data that are combinations of the user input and emotion values ​​indicating each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions that are located close to each other have similar values, as in the emotion map 900 shown in Fig. 10. Fig. 10 shows an example in which multiple emotions, "relief," "calm," and "encouraging," have similar emotion values.

[1976] In the above embodiment, an example was given in which a specific process is performed by one computer 22, but the technology disclosed herein is not limited to this, and distributed processing of the specific process may be performed by multiple computers, including computer 22.

[1977] In the above embodiment, an example in which the specific processing program 56 is stored in the storage 32 has been described, but the technology of the present disclosure is not limited to this. For example, the specific processing program 56 may be stored in a portable computer-readable non-transitory storage medium such as a Universal Serial Bus (USB) memory. The specific processing program 56 stored in the non-transitory storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes the specific processing in accordance with the specific processing program 56.

[1978] In addition, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 upon request from the data processing device 12.

[1979] It is not necessary to store all of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store all of the specific processing program 56 in the storage 32; only a portion of the specific processing program 56 may be stored.

[1980] As the hardware resource for executing the specific process, various processors as shown below can be used. An example of the processor is a CPU, which is a general-purpose processor that functions as a hardware resource for executing the specific process by executing software, i.e., a program. Another example of the processor is a dedicated electric circuit, which is a processor having a circuit configuration designed exclusively for executing the specific process, such as a Field-Programmable Gate Array (FPGA), a Programmable Logic Device (PLD), or an Application Specific Integrated Circuit (ASIC). Each processor has a built-in or connected memory, and each processor executes the specific process by using the memory.

[1981] The hardware resource that executes the specific process may be one of these various processors, or may be a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Also, the hardware resource that executes the specific process may be a single processor.

[1982] As an example of a configuration using one processor, first, one processor is configured by combining one or more CPUs and software, and this processor functions as a hardware resource that executes a specific process. Second, there is a configuration using a processor that realizes the functions of the entire system including multiple hardware resources that execute a specific process on a single IC chip, as typified by SoC (System-on-a-chip). In this way, a specific process is realized using one or more of the above-mentioned various processors as hardware resources.

[1983] Furthermore, more specifically, the hardware structure of these various processors can be an electric circuit that combines circuit elements such as semiconductor elements. The specific processes described above are merely examples. It goes without saying that unnecessary steps may be deleted, new steps may be added, or the order of processes may be changed without ...

Claims

1. means for obtaining a prompt statement describing specifications or characteristics of a digital asset desired by a user; A means for generating a digital asset such as image data or a 3D model using a generative AI model based on the prompt sentence; A means for placing the generated digital assets in a specific location within the metaverse; and A system including:

2. A means for acquiring a prompt statement indicating the content of a digital asset desired by a user; A means for obtaining a digital asset generated by a generative AI model by inputting the obtained prompt sentence into the generative AI model; a means for storing the generated digital assets together with their metadata in a database; means for allowing said user to download digital assets stored in said database in response to a request from said user; A system including:

3. A means for acquiring a prompt statement indicating the content and location of a digital asset desired by a user; A means for generating a digital asset using a generative AI model based on the prompt sentence; A means for placing the generated digital asset in a location instructed in the prompt sentence in the metaverse; A system including:

Images