System

Abstract

A system is provided.SOLUTION: A system comprising: means for allowing a user to select an item to be seen in a virtual space, a game, or a comic; means for setting customization options such as a color, a size, and a material for the item selected by the user; means for displaying an image generated based on the customization options; means for allowing the user to confirm the generated image and determine an order content; and means for linking the determined order content to an associated company to manufacture and deliver a product.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] In recent years, consumers who are inspired by media such as virtual spaces, games, and comics have increasingly expressed a desire to acquire specific items, outfits, or travel itineraries they have seen in the real world. However, it is often difficult to find ready-made products that match these specific images, resulting in a problem that cannot satisfy consumer expectations. Furthermore, the process of custom-making such items is complicated, and there is no established method for users to easily order and purchase customized products. [Means for solving the problem]

[0005] The present invention provides a system that allows users to easily select items found in virtual spaces, games, comics, etc., and set customization options such as color, size, and material. The system includes a means for the user to first select an item and then set customization options. It also includes a means for displaying to the user an image generated using an AI-generated model based on the customized options. The system also includes a means for the user to review the generated image and confirm the order details, a means for storing the confirmed order details in a database, and a means for linking with related vendors to have the product manufactured and delivered. This configuration allows users to easily order and purchase customized products, and enables vendors to earn revenue through a kickback.

[0006] A "user" is an individual or organization that uses a particular item or service.

[0007] A "virtual space" is an artificial environment created by computer simulation in which users can interact.

[0008] A "game" is electronic interactive software for entertainment purposes that allows players to play specific roles.

[0009] A "comic" is a form of sequential pictures that visually depicts a story, usually containing text or dialogue.

[0010] An "item" is a concrete object or concept that a user encounters in a virtual space, game, comic, etc.

[0011] "Customization options" are options that allow the user to freely select and set attributes such as the color, size, and material of an item.

[0012] An "AI generative model" is a system that uses artificial intelligence technology to generate images and data based on customization options set by the user.

[0013] "Image" means the visual representation of the customized item that is displayed for the user to view and review.

[0014] An "order content" is an order for fulfillment that includes detailed information and customization options for the item that the user has finalized.

[0015] "Affiliated vendors" are external companies or businesses that work together to manufacture and deliver items based on the user's order.

[0016] "Back margin" is a portion of the revenue that a system provider receives from related parties, and is paid as compensation for the provision of services. [Brief explanation of the drawings]

[0017] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8]FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 3 is a sequence diagram showing a processing flow of the data processing system according to the first embodiment. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0018] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0019] First, the terms used in the following description will be explained.

[0020] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0021] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0022] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0023] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0024] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0025] [First embodiment]

[0026] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0027] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0028] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0029] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0030] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0031] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0032] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0033] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0034] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0035] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0036] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0037] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0038] This invention relates to a system that allows users to customize and order items they see in virtual spaces, games, and comics, and then receive them as real-world products. This system allows users to freely customize specific items to their liking and easily order and purchase them. The program processing of this system is explained below with specific examples.

[0039] A user accesses the system

[0040] Terminal: The user accesses the system via a browser or a dedicated app.

[0041] Example: A user uses a smartphone to access the system's login page, enters their username and password, and presses the login button.

[0042] Server: Receives login information and performs authentication.

[0043] The server retrieves user information from the database, verifies the entered username and password, and if authentication is successful, returns the dashboard screen to the terminal.

[0044] The user selects the product they want

[0045] Terminal: User selects product category and chooses specific items.

[0046] Example: A user selects "game items" and then selects "sword model."

[0047] Server: Serves a list of products based on the selected category.

[0048] The server retrieves products related to the "game item" from the database and returns the list to the terminal, which then displays the product list to the user.

[0049] Set customization options

[0050] Terminal: The user sets the product's color, size, material, etc.

[0051] Example: A user sets the color of a "sword model" to silver, the length to 50 cm, and the material to wood.

[0052] Server: Receives and temporarily saves the configured customization options.

[0053] The server temporarily stores the received customization information in a database.

[0054] Gemini® Image Generation and Verification

[0055] Server: Sends customization information to Gemini AI and generates images.

[0056] The server calls the Gemini API based on the stored customization information to generate an image of the customized item.

[0057] Terminal: displays the generated image to the user.

[0058] Example: Image data sent from Gemini AI is displayed on the device, and the user checks the image.

[0059] Confirmation of order details

[0060] User: Check the generated image and confirm the order details.

[0061] If the user is satisfied with the image, he / she presses the "Confirm Order" button to confirm the order.

[0062] Server: Stores the order details in a database and generates information to be shared with related vendors.

[0063] The server officially stores the order details in a database and transmits the order information to related vendors.

[0064] Cooperation with vendors

[0065] Server: Sends order information to related vendors.

[0066] The server sends the order information via the vendor's API and waits for order confirmation from the vendor.

[0067] Supplier: Starts production based on the order information.

[0068] Example: A vendor receives an order and manufactures a model sword, preparing it for delivery.

[0069] User Notice and Receipt

[0070] Server: Notify the user that the product is complete.

[0071] The server receives a notification of production completion from the supplier and notifies the user via email or app notification.

[0072] Terminal: User receives notification and checks delivery status.

[0073] Users receive notifications and view progress on the delivery status page.

[0074] User: Receives the product and confirms receipt.

[0075] Example: After receiving the product, the user presses the receipt confirmation button on the system.

[0076] Closing process and back margin payment

[0077] Server: Calculates back margins, closes the transaction, and bills the sales company.

[0078] The server periodically extracts order and sales information from the database, calculates the back margin based on the contract with the related vendor, and issues invoices, which allows the system provider to earn revenue.

[0079] The processing flow will be explained below.

[0080] Step 1:

[0081] Terminal: The user accesses the system and moves to the login screen. They enter their username and password and press the login button.

[0082] Server: Receives login information, retrieves user information from the database, and performs authentication. After successful authentication, returns the user dashboard to the terminal.

[0083] Step 2:

[0084] Device: User clicks the "Choose a Product" button from the dashboard. The category selection page appears.

[0085] Device: The user selects a product category, for example, "Game Items."

[0086] Server: Based on the selected category, retrieves a list of related products from the database and returns it to the terminal, which then displays the product list to the user.

[0087] Step 3:

[0088] Terminal: The user selects a specific product from the list, such as a "model sword," and clicks the "Customize" button.

[0089] Device: A customization screen appears, allowing the user to enter customization options such as color, size, and material.

[0090] Server: Receives the entered customization information and stores it in a temporary database.

[0091] Step 4:

[0092] Server: Sends an image generation request to Gemini AI based on the saved customization information.

[0093] Server: Receives the image data of the customized product returned from Gemini AI and returns it to the terminal.

[0094] Terminal: Displays customized product images to the user.

[0095] Step 5:

[0096] Terminal: The user checks the generated image and clicks the "Confirm Order" button to confirm the order.

[0097] Server: Receives order confirmation information and stores it in the database as official order data. Generates order information for related vendors.

[0098] Step 6:

[0099] Server: Sends order information via the vendor's API or email system. Receives order confirmation from the vendor.

[0100] Vendor: Starts the manufacturing process of the product based on the received order information.

[0101] Step 7:

[0102] Server: Receives notification of product manufacturing completion from the supplier. Sends notification of product manufacturing completion to the user via email or app notification.

[0103] On the device: The user receives a notification and checks the delivery status of the product on the delivery status page.

[0104] Step 8:

[0105] User: Receives the product and presses the receipt confirmation button on the system.

[0106] Server: Saves the receipt confirmation information in a database.

[0107] Step 9:

[0108] Server: Periodically extracts order and sales information from the database, calculates back margins, and issues invoices to related vendors.

[0109] Example 1

[0110] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0111] Conventional online shopping systems make it difficult for users to customize and order digital items as real physical products. Furthermore, it is difficult for users to visually confirm the customized items, and there are many cases where the product does not meet expectations after ordering. Furthermore, there are problems with efficiently calculating back margins and billing procedures linked to the product manufacturing process.

[0112] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0113] In this invention, the server includes a means for allowing a user to select a digital item to be seen in a virtual space or entertainment content, a means for setting customization options such as color, size, and material for the selected digital item, and a means for displaying an image generated using a generative AI model based on the customization options. This allows a user to easily customize a digital item while visually checking it and then order it as a real physical product. Furthermore, by linking with the manufacturing process, back margin calculations and billing procedures can be efficiently performed.

[0114] "User" means an individual or entity that utilizes the System to select, customize, and order Digital Items as physical products.

[0115] "Digital items" refer to various products and objects that are visually recognized in virtual spaces and entertainment content.

[0116] "Customization options" are options that allow a user to change the color, size, material, etc. of a digital item selected by the user according to the user's preferences.

[0117] A "generative AI model" is an artificial intelligence algorithm for generating images of a digital item based on customization options.

[0118] "Image generation" is the process of using a generative AI model to create a visual display based on customization options.

[0119] A "manufacturing organization" is a company or factory that receives orders from users, manufactures physical products, and delivers them.

[0120] A "physical product" is a physical product that is created by physically creating a digital item.

[0121] "Back margin" refers to the commission or profit sharing that a system provider receives from a manufacturing organization based on a contract with the manufacturing organization for orders or sales.

[0122] "Notification means" refers to the method by which the system keeps users informed about product completion and delivery status.

[0123] A "prompt sentence" is text data input into a generative AI model, and contains customization information for image generation.

[0124] This invention relates to a system that allows users to select digital items that appear in virtual spaces or entertainment content, customize them, and order them as physical products. This system realizes a smooth process from customization to ordering and product delivery through mutual cooperation between the user, terminal, and server.

[0125] First, users access the system via a smartphone or computer browser or a dedicated app. They access the system's login page and log in by entering their username and password. After that, the dashboard screen is displayed, allowing the user to freely operate the system.

[0126] Next, the user selects a product category from the displayed dashboard and then selects a specific digital item from that category. For example, the user can select "Game Items" and then select a "Sword Model." For the selected digital item, the user can set customization options such as color, size, and material.

[0127] Once customization is complete, the server uses a generative AI model such as Gemini to generate an image based on the customization options. This generated image is then displayed to the user via their device. The user can review the displayed image and customize it again if necessary.

[0128] If the user is satisfied with the generated image, they press the "Confirm Order" button to confirm the order. The server officially saves this order in the database and shares the information with the manufacturing organization. At this point, the manufacturing organization begins manufacturing the physical product based on the received customization information.

[0129] Once the product is completed, the manufacturing organization sends the completion information to the server, and the server notifies the user. The user receives the notification and can check the delivery status. Once the product is delivered to the user, the user confirms receipt in the system and the process is complete. Finally, the server calculates the back margin based on the order information and bills the relevant organization.

[0130] For example, suppose a user selects a "model sword," sets the color to silver, the length to 50cm, and the material to wood, and requests that an image be generated. An example prompt for this would be:

[0131] Example prompt sentence:

[0132] "The user has selected a model sword and specified customization options: color is silver, length is 50cm, and material is wood. Use this information to generate an image of the customized model sword."

[0133] As a result, the system of the present invention provides an efficient and intuitive platform that allows users to easily customize digital items visually and order them as real physical products, while also realizing monetization for the system provider by automating the manufacturing process and back-margin calculation.

[0134] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0135] Step 1: User Login

[0136] Input: Username and Password

[0137] Specific operation: The user accesses the system's login page using a browser on their device (smartphone or computer) or a dedicated app, enters their username and password, and presses the login button.

[0138] Data processing and calculation: The server receives the entered username and password, retrieves the user information from the database, and performs verification.

[0139] Output: If the match is successful, the dashboard screen is returned to the terminal, allowing the user to operate the system.

[0140] Step 2: Select your product

[0141] Input: User selection of product category and specific item

[0142] Specific operation: The user selects a product category (e.g., "game items") from the dashboard and then selects a specific item (e.g., "sword model") from that category.

[0143] Data processing and calculation: The server receives the user's selection information and retrieves the corresponding product list from the database.

[0144] Output: The server returns the product list to the terminal, which displays the list to the user.

[0145] Step 3: Configure customization options

[0146] Input: Customization options such as color, size, material, etc. set by the user.

[0147] Specific operation: The user sets the color of the selected "sword model" to silver, the length to 50cm, and the material to wood.

[0148] Data processing and calculation: The server receives the customization options and temporarily stores them in the database.

[0149] Output: The customization information is saved in the database.

[0150] Step 4: Generate customized images

[0151] Input: Customization option information

[0152] Specific operation: The server calls the API of a generative AI model (e.g., Gemini) based on the saved customization information to generate an image of the customized item. It sends the following prompt: "The user selected a sword model and specified customization options. The color is silver, the length is 50 cm, and the material is wood. Please generate an image of the customized sword model based on this information."

[0153] Data processing and calculation: The server generates an image using the generative AI model and receives the image data.

[0154] Output: The generated image data is sent from the server to the terminal and displayed.

[0155] Step 5: Confirm your order

[0156] Input: User's order confirmation instructions

[0157] Specific operation: The user checks the generated image and presses the "Confirm Order" button if satisfied.

[0158] Data processing and calculation: The server officially stores the order details in the database and generates information linked to the manufacturing organization.

[0159] Output: The purchase order is sent to the manufacturing organization and production of the product begins.

[0160] Step 6: Product Completion Notification and Delivery Confirmation

[0161] Input: Product completion notification from manufacturing organization

[0162] Specific operation: The server receives a product completion notification from the manufacturing organization and notifies the user.

[0163] Data processing and calculation: The server sends the product completion information to the user via email or app notification.

[0164] Output: User receives notification and checks delivery status.

[0165] Step 7: Acknowledgement

[0166] Input: User's receipt confirmation instructions

[0167] Specific operation: After receiving the product, the user presses the receipt confirmation button on the system.

[0168] Data processing and calculation: The server saves the receipt confirmation data in the database.

[0169] Output: The receipt confirmation information is saved in the system.

[0170] Step 8: Calculate back margin

[0171] Input: Order and sales information

[0172] Specific operation: The server periodically extracts order and sales information from the database.

[0173] Data processing and calculation: The server calculates the back margin based on the order and sales information and issues an invoice.

[0174] Output: The calculation results and invoices are sent to the relevant organizations, and the system provider receives revenue.

[0175] (Application example 1)

[0176] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0177] In conventional systems where users receive items seen in virtual spaces, games, or comics as real-world products, it was sometimes difficult to manufacture and deliver products that accurately reflected the user's customizations. Furthermore, there was a lack of means for users to check the items they customized, and insufficient collaboration with vendors, making it difficult to improve the user experience. Furthermore, improving the customization experience in virtual stores using smart devices was also a challenge.

[0178] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0179] In this invention, the server includes a means for allowing a user to select an item seen in a virtual reality space, electronic game, or comic; a means for setting customization options, such as color, size, and material, for the selected item; and a means for displaying an image generated based on the customization options. This allows the user to browse a virtual store and customize the item via a smart device. The server also includes a means for confirming the generated image and confirming order details, and a means for linking the confirmed order details to a manufacturer and having the product manufactured and delivered. This ensures that products reflecting the user's customization are manufactured and delivered. Furthermore, the server includes a means for generating an image based on the customization options using a generative AI model and a means for using prompt sentences, enabling the server to quickly and accurately generate an image that accurately reflects the user's customization, improving the user experience.

[0180] "User" refers to an individual who selects and customizes an item seen in a virtual reality space, electronic game, or comic, and then checks the generated image and places an order.

[0181] A "virtual reality space" is a virtual environment created using computer or digital technology that users can interact with.

[0182] An "electronic game" is an interactive entertainment system that is implemented using digital devices.

[0183] "Manga" is content that combines narrative and illustrations using visual storytelling.

[0184] "Items" is a general term for items that users encounter in virtual reality spaces, electronic games, and comics.

[0185] "Customization options" refer to setting items that a user can specify for the item they have selected, including color, size, material, and the like.

[0186] "Generated Image" means a visual representation generated by an AI generative model based on customization options set by a user.

[0187] "Order details" are detailed information that allows the user to check the generated image and confirm the purchase.

[0188] A "manufacturer" is a company or organization that receives a user's order, manufactures the actual goods, and delivers them.

[0189] A "smart device" is an electronic device that can be carried by a user and that can connect to the Internet and run applications, and includes smartphones, smart glasses, etc.

[0190] A "virtual store" is a virtual shopping area built in a virtual reality space where users can browse, customize, and order items.

[0191] A "generative AI model" is an artificial intelligence algorithm used to generate an image of an item based on a user's customization options.

[0192] A "prompt" is textual input that instructs a generative AI model to generate a specific image.

[0193] This invention relates to a system that allows users to select an item they see in a virtual reality space, an electronic game, or a comic, customize it, and receive it as a real product. This system allows users to customize the item to their liking and easily order and purchase it. The program processing of this system is explained in detail below.

[0194] Hardware and Software Configuration

[0195] The system uses the following hardware and software:

[0196] Smart devices: Used by users to access and customize virtual spaces. Specifically, these include smartphones and smart glasses.

[0197] Server: Receives requests from users and performs authentication, data processing, image generation, and coordination of order details.

[0198] Flask: Used as a server-side web application framework.

[0199] MongoDB: Used as a database to store user information, product information, customization information, and order information.

[0200] Generative AI Model: The AI ​​model used to generate images of customized items. Specifically, it utilizes the Gemini AI API.

[0201] System processing flow

[0202] 1. User login: The user accesses the system using a smart device and enters their username and password on the login page for authentication. The server retrieves the user information from MongoDB and performs authentication.

[0203] 2. Product category selection: After successful authentication, the user selects a product category, for example, "Sword model" from the "Game items" category.

[0204] 3. Setting customization options: The user sets customization options for the selected item, such as color, size, material, etc. For example, the color of the "model sword" can be silver, the length can be 50 cm, and the material can be wood.

[0205] 4. Image Generation: The server calls the generative AI model (Gemini AI) based on the customization information and generates an image of the customized item. The generated image is displayed to the user.

[0206] 5. Confirmation of order: The user checks the generated image and confirms the order details. The server saves the order details in the database and generates information to be shared with related vendors.

[0207] 6. Linking to vendor: The server sends the order information via the vendor's API and waits for the order confirmation from the vendor. The vendor starts production based on the received order information.

[0208] 7. Notification and Delivery: The server receives the completed product and notifies the user via email or app notification. The user can check the progress on the card authentication page.

[0209] Specific prompt examples

[0210] Here are some example prompts to input to a generative AI model:

[0211] Generate images for the following customized items:

[0212] Item: Model sword

[0213] Color: Blue

[0214] Length: 60cm

[0215] Material: Iron

[0216] In this way, this system allows users to customize items they see in a virtual space and receive them as real products. By combining smart devices with advanced AI technology, it improves the user experience and realizes a smooth ordering, manufacturing, and delivery process.

[0217] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0218] Step 1:

[0219] A user accesses the system using a smart device and enters login information (user name and password).

[0220] Input: Username, Password

[0221] Data processing: The smart device sends the user name and password to the server, which searches for the corresponding user information in MongoDB and performs authentication.

[0222] Output: Authentication result (success / failure)

[0223] Specific operation: The server checks whether the username and password match, and if successful, displays the dashboard screen to the user.

[0224] Step 2:

[0225] The user selects a product category from the dashboard screen.

[0226] Input: Selected Product Category

[0227] Data processing: The selected category information is sent from the smart device to the server, which then retrieves the corresponding product list from MongoDB.

[0228] Output: Product list

[0229] Specific operation: The server displays the retrieved product list on the user's smart device, and the user selects a specific item from the list.

[0230] Step 3:

[0231] The user sets customization options (color, size, material, etc.) for the selected item.

[0232] Input: Customization options (color, size, material)

[0233] Data processing: Customization information is sent from the smart device to the server, which temporarily stores it in MongoDB.

[0234] Output: Save result (success / failure)

[0235] Specific operation: The server checks the received customization information, and if it is in the correct format, it temporarily saves it and proceeds to the next image generation process.

[0236] Step 4:

[0237] The server calls the generative AI model based on the saved customization information and generates an image of the customized item.

[0238] Input: Customization information

[0239] Data processing: The server sends the customization information as a prompt to the generative AI model, which then generates an image based on the customization information.

[0240] Output: The generated image

[0241] Specific operation: The server generates a prompt sentence in the following format and sends it to the generative AI model.

[0242] Generate images for the following customized items:

[0243] Item: Model sword

[0244] Color: Blue

[0245] Length: 60cm

[0246] Material: Iron

[0247] The generated image data is received and displayed to the user.

[0248] Step 5:

[0249] The user checks the generated image and confirms the order details.

[0250] Input: User confirmation (order confirmation)

[0251] Data processing: The user's confirmation is sent to the server. The server receives this, officially stores the order details in MongoDB, and prepares to send them to the relevant vendors.

[0252] Output: Order confirmation notice, order information

[0253] Specific operation: When the user presses the "Confirm Order" button, the server officially saves the order details and communicates with related vendors.

[0254] Step 6:

[0255] The server uses the vendor's API to send the order information.

[0256] Input: Order Information

[0257] Data processing: The server sends the order information to the vendor's API and receives an order confirmation.

[0258] Output: Order confirmation

[0259] Specific operation: The server sends the order information to the supplier's system and stores the order confirmation from the supplier in MongoDB.

[0260] Step 7:

[0261] The server receives the completion of the product and notifies the user.

[0262] Input: Notification of completion of production from supplier

[0263] Data processing: The server stores the received manufacturing completion notification in MongoDB and notifies the user via email or app notification.

[0264] Output: User notification

[0265] Specific operation: Based on the notification from the supplier, the server sends a notification to the user that the product has been completed, and the user can check the delivery status.

[0266] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0267] This invention combines an emotion engine with a system that allows users to customize and order items they see in virtual spaces, games, and comics, and then receive them as real-world products, thereby providing a customization and ordering process that responds to the user's emotions. Below, we will explain the program processing of this system using concrete examples.

[0268] A user accesses the system

[0269] Terminal: The user accesses the system via a browser or a dedicated app.

[0270] Example: A user uses a smartphone to access the system's login page, enters their username and password, and presses the login button.

[0271] Server: Receives login information, retrieves user information from the database, and performs authentication. After successful authentication, returns the user dashboard to the terminal.

[0272] The user selects the product they want

[0273] Device: The user clicks the "Select a product" button on the dashboard and is taken to the category selection page.

[0274] Example: A user selects "game items" and then selects "sword model."

[0275] Server: Based on the selected category, retrieves a list of related products from the database and returns it to the terminal, which then displays the product list to the user.

[0276] Set customization options

[0277] Device: The user inputs customization options such as color, size, material, etc. In addition, the emotion engine analyzes the user's emotions in real time and suggests customization options based on the user's preferences and emotions.

[0278] Example: A user sets the color of a "model sword" as silver, the length as 50 cm, and the material as wood, and consults the options suggested by the emotion engine.

[0279] Server: Receives the entered customization information and stores it in a temporary database.

[0280] Image generation and confirmation using Gemini

[0281] Server: Sends an image generation request to Gemini AI based on the saved customization information.

[0282] The server receives the image data of the customized product returned by Gemini AI and returns it to the terminal.

[0283] Terminal: The generated image is displayed to the user. The emotion engine analyzes the user's reaction in real time and readjusts the image as needed.

[0284] Example: Image data sent from Gemini AI is displayed on the device, and after the user checks the image, the emotion engine makes readjustments.

[0285] Confirmation of order details

[0286] User: Check the generated image and click the "Confirm Order" button to confirm the order.

[0287] The emotion engine displays recommendations based on the user's emotions.

[0288] Server: Receives order confirmation information and stores it in the database as official order data. Generates order information for related vendors.

[0289] Cooperation with vendors

[0290] Server: Sends order information via the vendor's API or email system. Receives order confirmation from the vendor.

[0291] The supplier starts the manufacturing process of the product based on the received order information.

[0292] User Notice and Receipt

[0293] Server: Receives notification of product manufacturing completion from the supplier. Sends notification of product manufacturing completion to the user via email or app notification.

[0294] On the device: The user receives a notification and checks the delivery status of the product on the delivery status page.

[0295] User: Receives the product and presses the receipt confirmation button on the system.

[0296] Example: After receiving the product, the user presses the receipt confirmation button in the system.

[0297] Closing process and back margin payment

[0298] Server: Periodically extracts order and sales information from the database, calculates back margins, and issues invoices to related vendors.

[0299] The server periodically calculates the back margin based on the order data and sends invoices to the relevant vendors. This process allows the system provider to earn revenue.

[0300] This allows users to easily select and confirm customized products that suit their preferences, and the emotion engine can provide an even more satisfying purchasing experience.

[0301] The processing flow will be explained below.

[0302] Step 1:

[0303] Terminal: The user accesses the system via a browser or a dedicated app, goes to the login screen, enters their username and password, and presses the login button.

[0304] Server: Receives login information, retrieves user information from the database, and performs authentication. After successful authentication, returns the user dashboard to the terminal.

[0305] Step 2:

[0306] Device: The user clicks the "Select a product" button on the dashboard and is taken to the category selection page.

[0307] Terminal: The user selects a product category, for example, "game items."

[0308] Server: Based on the selected category, retrieves a list of related products from the database and returns it to the terminal, which then displays the product list to the user.

[0309] Step 3:

[0310] Terminal: The user selects a specific product from the list, such as a "model sword," and clicks the "Customize" button.

[0311] Device: A customization screen is displayed, allowing the user to input customization options such as color, size, material, etc. Furthermore, an emotion engine analyzes the user's emotions in real time and suggests customization options based on the user's preferences and emotions.

[0312] Server: Receives the entered customization information and stores it in a temporary database.

[0313] Step 4:

[0314] Server: Sends an image generation request to Gemini AI based on the saved customization information.

[0315] Server: Receives the image data of the customized product returned from Gemini AI and returns it to the terminal.

[0316] Terminal: The generated image is displayed to the user. The emotion engine analyzes the user's reaction in real time and readjusts the image as needed.

[0317] Example: Image data sent from Gemini AI is displayed on the device, and after the user checks the image, the emotion engine makes readjustments.

[0318] Step 5:

[0319] Terminal: The user checks the generated image and clicks the "Confirm Order" button to confirm the order. The emotion engine displays recommended content based on the user's emotions.

[0320] Server: Receives order confirmation information and stores it in the database as official order data. Generates order information for related vendors.

[0321] Step 6:

[0322] Server: Sends order information via the vendor's API or email system. Receives order confirmation from the vendor.

[0323] Vendor: Starts the manufacturing process of the product based on the received order information.

[0324] Step 7:

[0325] Server: Receives notification of product manufacturing completion from the supplier. Sends notification of product manufacturing completion to the user via email or app notification.

[0326] On the device: The user receives a notification and checks the delivery status of the product on the delivery status page.

[0327] Step 8:

[0328] User: Receives the product and presses the receipt confirmation button on the system.

[0329] Server: Saves the receipt confirmation information in a database.

[0330] Step 9:

[0331] Server: Periodically extracts order and sales information from the database, calculates back margins, and issues invoices to related vendors.

[0332] Example 2

[0333] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0334] Currently, when users customize and order real-world products based on items they see in virtual spaces, electronic games, or electronic comics, it is difficult to achieve high satisfaction because there is a lack of suggestions that reflect the user's emotions and preferences. Another problem is that the purchase process is hindered by the cumbersome process of readjusting the customized images after viewing them.

[0335] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0336] In this invention, the server includes means for allowing a user to select an item seen in a virtual space, electronic game, or electronic comic, means for setting customization options such as color, size, and material for the item selected by the user, means for displaying an image generated using a generative AI model based on the customization options, means for the user to check the generated image and confirm the order details, means for coordinating the confirmed order details with related vendors to manufacture and deliver the product, and means for analyzing the user's emotions in real time and suggesting customization options according to the emotions. This allows users to smoothly customize, check, and order products according to their emotions and preferences.

[0337] "User" refers to an individual or group that uses the system to customize and order items found in virtual spaces, electronic games, and electronic comics as real-world products.

[0338] "Virtual space" refers to a virtual environment that is different from the real world and is constructed on the Internet or a computer network.

[0339] An "electronic game" is a type of interactive electronic entertainment played using a computer or gaming console.

[0340] "Digital comics" are manga and graphic novels available in digital format.

[0341] "Items" refer to items or objects that users encounter in virtual spaces, electronic games, and electronic comics.

[0342] "Customization options" refers to the choices and settings such as color, size, material, etc. that a user sets for an item.

[0343] A "generative AI model" is a model that uses artificial intelligence technology to generate images and data based on customization options selected by the user.

[0344] The "emotion engine" is part of a system that analyzes users' emotions in real time and provides customization options and suggestions based on those emotions.

[0345] "Affiliated Businesses" refers to companies or corporations that receive orders confirmed by users and manufacture and deliver products.

[0346] "Order details" refers to details of the item customized by the user, and includes specific instructions and specifications for manufacturing and delivery.

[0347] "Back margin" refers to a commission or margin on sales that is paid by related parties to the system provider.

[0348] The present invention relates to a system that allows users to customize items found in virtual spaces, electronic games, and electronic comics to their liking and receive them as real-world products. This system incorporates an emotion engine that analyzes the user's emotions in real time, and provides customization options and suggestions based on the emotions. A specific implementation of the system is described below.

[0349] System configuration and hardware / software usage examples

[0350] This system mainly consists of three components: a server, a terminal, and a user. The server includes a database, a public web server, an API server, and a generative AI model. The terminal is a smartphone, tablet, or personal computer that users access. The emotion engine is a software module that performs real-time emotion analysis of user input data.

[0351] A user accesses the system

[0352] Terminal: The user accesses the system via a smartphone or PC browser, or a dedicated app. The user accesses the system's login page, enters their username and password, and presses the login button. Built-in security features ensure data safety.

[0353] Example: A user accesses "example.com / login", enters the username "user123" and password "password123", and clicks the login button.

[0354] Server: Receives login information and retrieves the corresponding user information from the database. It compares the retrieved information with the entered information, and if authentication is successful, generates the user's dashboard and returns it to the device.

[0355] Product Selection

[0356] Device: The user clicks the "Select a product" button on the dashboard to go to the category selection page, where they can select the desired product from the list of product categories offered.

[0357] Example: A user clicks the "Choose a Product" button on the dashboard and selects "Gaming Items" on the category page.

[0358] Server: Based on the selected category, retrieves a list of related products from the database and returns it to the terminal, where the user can view the product list.

[0359] Setting customization options

[0360] Terminal: The user inputs customization options such as color, size, material, etc. The emotion engine analyzes the user's emotions in real time and suggests customization options based on their preferences and emotions.

[0361] Example: A user sets the color of a "model sword" to "silver," the length to "50cm," and the material to "wood." The emotion engine analyzes the user's input and presents additional customization options.

[0362] Server: Receives the entered customization information and stores it in a temporary database.

[0363] Image generation and confirmation

[0364] Server: Based on the saved customization information, the server sends an image generation request to the generative AI model. The generative AI model generates an image that reflects the customization options and sends it back to the server.

[0365] Terminal: The generated image is displayed to the user. The emotion engine analyzes the user's reactions in real time and makes suggestions to readjust the image if necessary.

[0366] Example: A customized image sent from the generative AI model is displayed on the device, and the user can review it. The emotion engine will make adjustments as necessary.

[0367] Confirmation of order details

[0368] User: Check the generated image and if satisfied, click the "Confirm Order" button. The emotion engine may also display recommendations based on the user's emotions.

[0369] Server: Receives order confirmation information and stores it in the database as official order data. Generates order information for related vendors and communicates it to the vendors.

[0370] Cooperation with vendors

[0371] Server: Sends order information via the vendor's API or email system and receives order confirmation from the vendor.

[0372] Example: Use a vendor API to send order information and get order confirmation from the vendor.

[0373] User Notice and Receipt

[0374] Server: Receives notification of product manufacturing completion from the supplier and sends notification of product manufacturing completion to the user via email or app notification.

[0375] On the device: The user receives a notification and checks the product delivery status on the delivery status page.

[0376] User: After receiving the product, press the confirmation button on the system.

[0377] Example: After a user receives a product, they press the "Confirm Receipt" button in the system.

[0378] Closing process and back margin payment

[0379] Server: Periodically extracts order and sales information from the database, calculates back margins, and issues invoices to related vendors.

[0380] Example: The server calculates the back margin based on the order data and sends an invoice to the relevant vendor.

[0381] Prompt Sentence Examples

[0382] "Design a system that allows users to customize items using an emotion engine and experience the ordering process. Required functionality includes login, product selection, customization options, image generation using generative AI, ordering, vendor integration, notifications, receipt confirmation, and closing."

[0383] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0384] Step 1: Access the system

[0385] Terminal: The user accesses the system using a smartphone or PC browser, or a dedicated app. The user enters their username and password on the login page and presses the login button.

[0386] Input: Username "user123", Password "password123"

[0387] Output: User dashboard

[0388] Specific operation: The user accesses "example.com / login", enters the username and password, and clicks the login button.

[0389] Server: Receives login information and retrieves the corresponding user information from the database. The retrieved information is compared with the entered information, and if authentication is successful, a user dashboard is generated and returned to the terminal.

[0390] Input: Login information (username and password)

[0391] Output: Authentication results and user dashboard

[0392] Specific operation: The server retrieves user information from the database, performs authentication, and if successful, creates and returns a user dashboard.

[0393] Step 2: Select product category

[0394] On the device: The user clicks the "Select a product" button on the dashboard, which takes them to the category selection page. The user then selects the desired category from the list of product categories provided.

[0395] Input: Category selection request

[0396] Output: Product category page

[0397] Specific behavior: The user clicks the "Choose a Product" button on the dashboard and selects the "Game Items" category.

[0398] Server: Based on the selected category, retrieves a list of related products from the database and returns it to the terminal, which then displays the product list to the user.

[0399] Input: Category selection information

[0400] Output: Product list

[0401] Specific operation: The server retrieves data in the "game items" category from the database, sends a product list to the user's terminal, and displays it.

[0402] Step 3: Configure customization options

[0403] Terminal: The user inputs customization options such as color, size, material, etc. The emotion engine analyzes the user's emotions in real time and suggests customization options based on the user's preferences and emotions.

[0404] Input: Customization option information

[0405] Output: Optimized customization options

[0406] What it does: The user sets the color of the "model sword" to "silver," the length to "50cm," and the material to "wood." The emotion engine uses these inputs to suggest additional customization options.

[0407] Server: Receives the entered customization information and stores it in a temporary database.

[0408] Input: User-entered customization information

[0409] Output: Save results to a temporary database

[0410] Specific operation: The server receives the customization information of "silver, 50cm, wood" and stores it in a temporary database.

[0411] Step 4: Image generation and verification

[0412] Server: Based on the saved customization information, the server sends an image generation request to the generative AI model. The generative AI model generates an image that reflects the customization options and sends it back to the server.

[0413] Input: Customization information

[0414] Output: Generated customized image

[0415] Specific operation: The server requests the generative AI model to generate an image of a "silver, 50 cm, wooden sword" and receives the generated image data.

[0416] Terminal: The generated image is displayed to the user. The emotion engine analyzes the user's reactions in real time and makes suggestions to readjust the image if necessary.

[0417] Input: Customized image data

[0418] Output: Optimized customized image

[0419] Specific operation: The image sent from the generative AI model is displayed on the device, and the emotion engine suggests readjustments based on the user's emotional response.

[0420] Step 5: Confirm your order

[0421] User: Check the generated image and if satisfied, click the "Confirm Order" button. The emotion engine may also display recommendations based on the user's emotions.

[0422] Input: Order confirmation request

[0423] Output: Order confirmation notification

[0424] Specific operation: The user checks the image and clicks the "Confirm Order" button.

[0425] Server: Receives order confirmation information and stores it in the database as official order data. Generates order information for related vendors and communicates it to the vendors.

[0426] Input: Order confirmation information

[0427] Output: Official order data and order information to vendors

[0428] Specific operation: The server saves the order information in a database and prepares to send the order information to the supplier.

[0429] Step 6: Contact a vendor

[0430] Server: Sends order information via the vendor's API or email system and receives order confirmation from the vendor.

[0431] Input: Order data

[0432] Output: Vendor order confirmation

[0433] Specific operation: The server sends the order information to the supplier and receives an order confirmation.

[0434] Step 7: User Notification and Acceptance

[0435] Server: Receives notification from the supplier that product manufacturing is complete and sends notification of product manufacturing completion to the user via email or app notification.

[0436] Input: Product manufacturing completion notification

[0437] Output: Product manufacturing completion notification to user

[0438] Specific operation: The server receives the product manufacturing completion information and sends a notification to the user.

[0439] On the device: The user receives a notification and checks the product delivery status on the delivery status page.

[0440] Input: Receive completion notification

[0441] Output: Display delivery status

[0442] What happens: A user checks the delivery status of a product online.

[0443] User: After receiving the product, press the confirmation button on the system.

[0444] Input: Receipt Acknowledgment Request

[0445] Output: Notification of receipt

[0446] Specific operation: The user receives the product and clicks the "Confirm receipt" button.

[0447] Step 8: Closing and margin payment

[0448] Server: Periodically extracts order and sales information from the database, calculates back margins, and issues invoices to related vendors.

[0449] Input: Order and sales data

[0450] Output: Back margin calculation results and invoice

[0451] Specific operation: The server calculates the back margin based on the order data for the specified period and sends an invoice to the relevant vendor.

[0452] (Application example 2)

[0453] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0454] In current systems that allow users to receive items seen in virtual spaces or games as physical goods in the real world, the customization experience is complicated, making it difficult to achieve highly satisfying customization that reflects the user's emotions.Another issue is that the wide range of customization options makes it difficult for users to make the optimal choice.

[0455] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0456] In this invention, the server includes: a means for allowing a user to select an item seen in a virtual space, game, or comic; a means for setting customization options such as color, size, and material for the item selected by the user; an emotion analysis means for analyzing the user's emotions in real time and suggesting customization options; a means for displaying an image generated based on the customization options; a means for the user to check the generated image and confirm the order details; and a means for coordinating the confirmed order details with related vendors to have the product manufactured and delivered. This enables a highly satisfying customization and ordering process that reflects the user's emotions.

[0457] A "virtual space" is a virtual environment generated by a computer.

[0458] A "game" is an interactive activity under prescribed rules for recreational or competitive purposes.

[0459] "Comics" is a form of publication that tells a story through pictures and text.

[0460] An "item" is a thing or element that exists in relation to a particular use or purpose.

[0461] "Customization options" are options that allow users to set the color, size, material, etc. of an item according to their preferences.

[0462] "Emotion analysis means" is a technology that analyzes emotions in real time from a user's facial expressions and behavior.

[0463] A "generated image" is a visual representation generated based on the customization options set by the user.

[0464] "Order details" refers to product specifications and order information confirmed by the user.

[0465] "Affiliated businesses" are companies or organizations that manufacture and deliver products based on user orders.

[0466] A "generative AI model" is an algorithm that uses artificial intelligence to generate images or text from data.

[0467] A "system" is a set of processes or devices consisting of multiple elements configured to achieve a specific purpose.

[0468] This invention provides a customization and ordering process that responds to the user's emotions by combining an emotion analysis engine with a system that allows users to customize items they see in virtual spaces, games, and comics and receive them as real products. The following describes an embodiment of the invention.

[0469] Hardware and software used

[0470] 1. Smartphone: The user uses an iOS or ANDROID (registered trademark) device. A dedicated application is installed on the smartphone.

[0471] 2. Emotion Analysis Engine: Affectiva SDK is used. This SDK analyzes the user's facial expression data and grasps their emotional state in real time.

[0472] 3. Generative AI models: OpenAI® GPT and Gemini AI are used. OpenAI GPT analyzes and generates natural language, and Gemini AI generates images based on customization options.

[0473] 4. Server: The server uses AWS (registered trademark) (Amazon Web Services), which handles back-end processing, data storage, and vendor integration.

[0474] 5. Database: MySQL (registered trademark) and DynamoDB are used to store customization information, order information, user information, etc.

[0475] 6. Notification system: Use Firebase Cloud Messaging (FCM) to send notifications to users.

[0476] 7. API communication: Data communication between systems is carried out using RESTful APIs.

[0477] Specific examples of the invention

[0478] A user accesses the system

[0479] Users access the system from a smartphone application and log in by entering their username and password. The server receives the login information, retrieves user information from the database, and performs authentication. If authentication is successful, it returns the user dashboard to the device.

[0480] The user selects the product they want

[0481] The user clicks the "Select a Product" button on the smartphone dashboard, which takes them to a category selection page. The user selects "Game Items" and then "Model Sword" from the list. The server retrieves a list of related products from the database based on the selected category and returns it to the device. The user then checks the product list on their smartphone.

[0482] Set customization options

[0483] Users input customization options such as color, size, and material. At this time, the sentiment analysis engine analyzes the user's emotions in real time and suggests customization options based on the user's preferences and emotions. For example, if a user sets the color of a "model sword" as silver, the length as 50 cm, and the material as wood, the sentiment analysis engine will suggest better options.

[0484] Image generation and confirmation using Gemini

[0485] The server sends an image generation request to Gemini AI, a generative AI model, based on the saved customization information. The image data of the customized product returned by Gemini AI is then received and returned to the device. The user can then view the generated image on their smartphone, and the sentiment analysis engine analyzes the user's reactions in real time, readjusting the image as necessary.

[0486] Confirmation of order details

[0487] The user checks the generated image and clicks the "Confirm Order" button to confirm the order. At this time, the sentiment analysis engine displays recommendations based on the user's emotions. The server receives the order confirmation information and stores it in the database as official order data. It also generates order information for related vendors.

[0488] Cooperation with vendors

[0489] The server sends the order information via the supplier's API or email system, receives an order confirmation from the supplier, and the supplier starts the manufacturing process based on the order information.

[0490] User Notice and Receipt

[0491] When the server receives a notification from the supplier that the product has been completed, it will notify the user by email or app notification. The user will receive the notification and check the product's delivery status on the delivery status page. When the user receives the product, they can press the receipt confirmation button on the system to complete the receipt.

[0492] Closing process and back margin payment

[0493] The server periodically extracts order and sales information from the database, calculates the back margin, and issues invoices to related vendors, allowing the system provider to earn revenue.

[0494] Prompt Sentence Examples

[0495] Below is an example prompt that can be input to a generative AI model to generate suggested customization options based on sentiment analysis.

[0496] text

[0497] The user selected the "Sword Model." Based on the user's facial expression data, their current emotional state is "Happy." Suggest the best customization options for this sword model based on the emotion of "Happy." Recommend color, material, and design changes as needed.

[0498] This prompt provides emotional data to the generative AI model, which then suggests appropriate customization options. This approach allows users to customize products in a way that best suits their emotions, resulting in a satisfying shopping experience.

[0499] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0500] Step 1:

[0501] A user accesses the system and enters their login information.

[0502] Input: Username and Password

[0503] Specific operation: The user accesses the login screen of the smartphone app, enters their username and password, and presses the login button.

[0504] Data processing: The server receives the entered login information and retrieves user information from the database (MySQL).

[0505] Output: If authentication is successful, returns the user dashboard to the device.

[0506] Step 2:

[0507] The user selects the product they want.

[0508] Input: "Select a product" button and category selection (e.g., game items, sword model)

[0509] Specific Action: The user clicks the "Select a Product" button on the dashboard, goes to the category selection page, and then selects a specific item (e.g., "Model Sword") from the selected category.

[0510] Data processing: The server retrieves a list of related products from the database (DynamoDB) based on the selected category.

[0511] Output: The terminal displays the product list to the user.

[0512] Step 3:

[0513] Enter the customization options.

[0514] Input: Customization information such as item color, size, material, etc.

[0515] How it works: Users input customization options for the selected item, such as color, size, material, etc. At the same time, the emotion analysis engine (Affectiva SDK) analyzes the user's facial expression data in real time.

[0516] Data processing: The server sends the sentiment analysis data to OpenAI GPT, which then suggests customization options based on the user's emotions.

[0517] Output: Prints the suggested customization options to the terminal.

[0518] Step 4:

[0519] Check the generated image.

[0520] Input: Customization information and user emotion data

[0521] Specific operation: The user confirms the settings based on the customization options. After the settings are confirmed, the server sends an image generation request to Gemini AI based on the settings.

[0522] Data processing: Gemini AI creates a generated image based on the customization information and sends it back to the server.

[0523] Output: The server returns the generated image to the device, which displays it to the user. The sentiment analysis engine then analyzes the user's reaction again and adjusts the image if necessary.

[0524] Step 5:

[0525] Confirm the order details.

[0526] Input: Confirmed customizations and generated images

[0527] Specific operation: The user checks the generated image and clicks the "Confirm Order" button. At this time, the sentiment analysis engine displays recommendations based on the user's emotions.

[0528] Data processing: The server receives the order confirmation information, stores it in the database as official order data, and generates order information for related vendors.

[0529] Output: The order information is sent to the relevant supplier and the manufacturing process is initiated.

[0530] Step 6:

[0531] Product manufacturing and user notification.

[0532] Input: Production completion notice from supplier

[0533] Specific operation: The server receives notification from the relevant supplier that the product has been completed. Based on this information, it uses Firebase Cloud Messaging (FCM) to notify the user that the product has been completed.

[0534] Data processing: Generates notification information and sends it to the user's device.

[0535] Output: The user receives a notification on their smartphone that production is complete and checks the delivery status on the delivery status page.

[0536] Step 7:

[0537] Receipt and confirmation in the system.

[0538] Input: When the user receives the product, they press the "Confirm Receipt" button.

[0539] Specific operation: After receiving the product, the user presses the "Confirm receipt" button on the smartphone app, which causes the system to confirm receipt.

[0540] Data processing: The server records the received data and stores it in a database.

[0541] Output: The receipt confirmation information is saved in the database and the product receipt procedure is completed.

[0542] Step 8:

[0543] Regular data processing and back margin calculation.

[0544] Input: Order and sales information in the database

[0545] Specific operation: The server periodically extracts order information and sales information from the database, calculates the back margin based on this information, and issues invoices to related vendors.

[0546] Data processing: Analyze and calculate sales data and order information.

[0547] Output: Invoices are sent to the relevant vendors and back margin payments are processed.

[0548] As described above, by clearly indicating the specific operations, inputs, and outputs at each step, the user can smoothly complete the customization and ordering process and achieve a high level of satisfaction.

[0549] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0550] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (registered trademark) (Internet search engine).<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0551] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0552] [Second embodiment]

[0553] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0554] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0555] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0556] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0557] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0558] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0559] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0560] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0561] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0562] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0563] In the smart glasses 214, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0564] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0565] This invention relates to a system that allows users to customize and order items they see in virtual spaces, games, and comics, and then receive them as real-world products. This system allows users to freely customize specific items to their liking and easily order and purchase them. The program processing of this system is explained below with specific examples.

[0566] A user accesses the system

[0567] Terminal: The user accesses the system via a browser or a dedicated app.

[0568] Example: A user uses a smartphone to access the system's login page, enters their username and password, and presses the login button.

[0569] Server: Receives login information and performs authentication.

[0570] The server retrieves user information from the database, verifies the entered username and password, and if authentication is successful, returns the dashboard screen to the terminal.

[0571] The user selects the product they want

[0572] Terminal: User selects product category and chooses specific items.

[0573] Example: A user selects "game items" and then selects "sword model."

[0574] Server: Serves a list of products based on the selected category.

[0575] The server retrieves products related to the "game item" from the database and returns the list to the terminal, which then displays the product list to the user.

[0576] Set customization options

[0577] Terminal: The user sets the product's color, size, material, etc.

[0578] Example: A user sets the color of a "sword model" to silver, the length to 50 cm, and the material to wood.

[0579] Server: Receives and temporarily saves the configured customization options.

[0580] The server temporarily stores the received customization information in a database.

[0581] Image generation and confirmation using Gemini

[0582] Server: Sends customization information to Gemini AI and generates images.

[0583] The server calls the Gemini API based on the stored customization information to generate an image of the customized item.

[0584] Terminal: displays the generated image to the user.

[0585] Example: Image data sent from Gemini AI is displayed on the device, and the user checks the image.

[0586] Confirmation of order details

[0587] User: Check the generated image and confirm the order details.

[0588] If the user is satisfied with the image, he / she presses the "Confirm Order" button to confirm the order.

[0589] Server: Stores the order details in a database and generates information to be shared with related vendors.

[0590] The server officially stores the order details in a database and transmits the order information to related vendors.

[0591] Cooperation with vendors

[0592] Server: Sends order information to related vendors.

[0593] The server sends the order information via the vendor's API and waits for order confirmation from the vendor.

[0594] Supplier: Starts production based on the order information.

[0595] Example: A vendor receives an order and manufactures a model sword, preparing it for delivery.

[0596] User Notice and Receipt

[0597] Server: Notify the user that the product is complete.

[0598] The server receives a notification of production completion from the supplier and notifies the user via email or app notification.

[0599] Terminal: User receives notification and checks delivery status.

[0600] Users receive notifications and view progress on the delivery status page.

[0601] User: Receives the product and confirms receipt.

[0602] Example: After receiving the product, the user presses the receipt confirmation button on the system.

[0603] Closing process and back margin payment

[0604] Server: Calculates back margins, closes the transaction, and bills the sales company.

[0605] The server periodically extracts order and sales information from the database, calculates the back margin based on the contract with the related vendor, and issues invoices, which allows the system provider to earn revenue.

[0606] The processing flow will be explained below.

[0607] Step 1:

[0608] Terminal: The user accesses the system and moves to the login screen. They enter their username and password and press the login button.

[0609] Server: Receives login information, retrieves user information from the database, and performs authentication. After successful authentication, returns the user dashboard to the terminal.

[0610] Step 2:

[0611] Device: User clicks the "Choose a Product" button from the dashboard. The category selection page appears.

[0612] Device: The user selects a product category, for example, "Game Items."

[0613] Server: Based on the selected category, retrieves a list of related products from the database and returns it to the terminal, which then displays the product list to the user.

[0614] Step 3:

[0615] Terminal: The user selects a specific product from the list, such as a "model sword," and clicks the "Customize" button.

[0616] Device: A customization screen appears, allowing the user to enter customization options such as color, size, and material.

[0617] Server: Receives the entered customization information and stores it in a temporary database.

[0618] Step 4:

[0619] Server: Sends an image generation request to Gemini AI based on the saved customization information.

[0620] Server: Receives the image data of the customized product returned from Gemini AI and returns it to the terminal.

[0621] Terminal: Displays customized product images to the user.

[0622] Step 5:

[0623] Terminal: The user checks the generated image and clicks the "Confirm Order" button to confirm the order.

[0624] Server: Receives order confirmation information and stores it in the database as official order data. Generates order information for related vendors.

[0625] Step 6:

[0626] Server: Sends order information via the vendor's API or email system. Receives order confirmation from the vendor.

[0627] Vendor: Starts the manufacturing process of the product based on the received order information.

[0628] Step 7:

[0629] Server: Receives notification of product manufacturing completion from the supplier. Sends notification of product manufacturing completion to the user via email or app notification.

[0630] On the device: The user receives a notification and checks the delivery status of the product on the delivery status page.

[0631] Step 8:

[0632] User: Receives the product and presses the receipt confirmation button on the system.

[0633] Server: Saves the receipt confirmation information in a database.

[0634] Step 9:

[0635] Server: Periodically extracts order and sales information from the database, calculates back margins, and issues invoices to related vendors.

[0636] Example 1

[0637] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0638] Conventional online shopping systems make it difficult for users to customize and order digital items as real physical products. Furthermore, it is difficult for users to visually confirm the customized items, and there are many cases where the product does not meet expectations after ordering. Furthermore, there are problems with efficiently calculating back margins and billing procedures linked to the product manufacturing process.

[0639] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0640] In this invention, the server includes a means for allowing a user to select a digital item to be seen in a virtual space or entertainment content, a means for setting customization options such as color, size, and material for the selected digital item, and a means for displaying an image generated using a generative AI model based on the customization options. This allows a user to easily customize a digital item while visually checking it and then order it as a real physical product. Furthermore, by linking with the manufacturing process, back margin calculations and billing procedures can be efficiently performed.

[0641] "User" means an individual or entity that utilizes the System to select, customize, and order Digital Items as physical products.

[0642] "Digital items" refer to various products and objects that are visually recognized in virtual spaces and entertainment content.

[0643] "Customization options" are options that allow a user to change the color, size, material, etc. of a digital item selected by the user according to the user's preferences.

[0644] A "generative AI model" is an artificial intelligence algorithm for generating images of a digital item based on customization options.

[0645] "Image generation" is the process of using a generative AI model to create a visual display based on customization options.

[0646] A "manufacturing organization" is a company or factory that receives orders from users, manufactures physical products, and delivers them.

[0647] A "physical product" is a physical product that is created by physically creating a digital item.

[0648] "Back margin" refers to the commission or profit sharing that a system provider receives from a manufacturing organization based on a contract with the manufacturing organization for orders or sales.

[0649] "Notification means" refers to the method by which the system keeps users informed about product completion and delivery status.

[0650] A "prompt sentence" is text data input into a generative AI model, and contains customization information for image generation.

[0651] This invention relates to a system that allows users to select digital items that appear in virtual spaces or entertainment content, customize them, and order them as physical products. This system realizes a smooth process from customization to ordering and product delivery through mutual cooperation between the user, terminal, and server.

[0652] First, users access the system via a smartphone or computer browser or a dedicated app. They access the system's login page and log in by entering their username and password. After that, the dashboard screen is displayed, allowing the user to freely operate the system.

[0653] Next, the user selects a product category from the displayed dashboard and then selects a specific digital item from that category. For example, the user can select "Game Items" and then select a "Sword Model." For the selected digital item, the user can set customization options such as color, size, and material.

[0654] Once customization is complete, the server uses a generative AI model such as Gemini to generate an image based on the customization options. This generated image is then displayed to the user via their device. The user can review the displayed image and customize it again if necessary.

[0655] If the user is satisfied with the generated image, they press the "Confirm Order" button to confirm the order. The server officially saves this order in the database and shares the information with the manufacturing organization. At this point, the manufacturing organization begins manufacturing the physical product based on the received customization information.

[0656] Once the product is completed, the manufacturing organization sends the completion information to the server, and the server notifies the user. The user receives the notification and can check the delivery status. Once the product is delivered to the user, the user confirms receipt in the system and the process is complete. Finally, the server calculates the back margin based on the order information and bills the relevant organization.

[0657] For example, suppose a user selects a "model sword," sets the color to silver, the length to 50cm, and the material to wood, and requests that an image be generated. An example prompt for this would be:

[0658] Example prompt sentence:

[0659] "The user has selected a model sword and specified customization options: color is silver, length is 50cm, and material is wood. Use this information to generate an image of the customized model sword."

[0660] As a result, the system of the present invention provides an efficient and intuitive platform that allows users to easily customize digital items visually and order them as real physical products, while also realizing monetization for the system provider by automating the manufacturing process and back-margin calculation.

[0661] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0662] Step 1: User Login

[0663] Input: Username and Password

[0664] Specific operation: The user accesses the system's login page using a browser on their device (smartphone or computer) or a dedicated app, enters their username and password, and presses the login button.

[0665] Data processing and calculation: The server receives the entered username and password, retrieves the user information from the database, and performs verification.

[0666] Output: If the match is successful, the dashboard screen is returned to the terminal, allowing the user to operate the system.

[0667] Step 2: Select your product

[0668] Input: User selection of product category and specific item

[0669] Specific operation: The user selects a product category (e.g., "game items") from the dashboard and then selects a specific item (e.g., "sword model") from that category.

[0670] Data processing and calculation: The server receives the user's selection information and retrieves the corresponding product list from the database.

[0671] Output: The server returns the product list to the terminal, which displays the list to the user.

[0672] Step 3: Configure customization options

[0673] Input: Customization options such as color, size, material, etc. set by the user.

[0674] Specific operation: The user sets the color of the selected "sword model" to silver, the length to 50cm, and the material to wood.

[0675] Data processing and calculation: The server receives the customization options and temporarily stores them in the database.

[0676] Output: The customization information is saved in the database.

[0677] Step 4: Generate customized images

[0678] Input: Customization option information

[0679] Specific operation: The server calls the API of a generative AI model (e.g., Gemini) based on the saved customization information to generate an image of the customized item. It sends the following prompt: "The user selected a sword model and specified customization options. The color is silver, the length is 50 cm, and the material is wood. Please generate an image of the customized sword model based on this information."

[0680] Data processing and calculation: The server generates an image using the generative AI model and receives the image data.

[0681] Output: The generated image data is sent from the server to the terminal and displayed.

[0682] Step 5: Confirm your order

[0683] Input: User's order confirmation instructions

[0684] Specific operation: The user checks the generated image and presses the "Confirm Order" button if satisfied.

[0685] Data processing and calculation: The server officially stores the order details in the database and generates information linked to the manufacturing organization.

[0686] Output: The purchase order is sent to the manufacturing organization and production of the product begins.

[0687] Step 6: Product Completion Notification and Delivery Confirmation

[0688] Input: Product completion notification from manufacturing organization

[0689] Specific operation: The server receives a product completion notification from the manufacturing organization and notifies the user.

[0690] Data processing and calculation: The server sends the product completion information to the user via email or app notification.

[0691] Output: User receives notification and checks delivery status.

[0692] Step 7: Acknowledgement

[0693] Input: User's receipt confirmation instructions

[0694] Specific operation: After receiving the product, the user presses the receipt confirmation button on the system.

[0695] Data processing and calculation: The server saves the receipt confirmation data in the database.

[0696] Output: The receipt confirmation information is saved in the system.

[0697] Step 8: Calculate back margin

[0698] Input: Order and sales information

[0699] Specific operation: The server periodically extracts order and sales information from the database.

[0700] Data processing and calculation: The server calculates the back margin based on the order and sales information and issues an invoice.

[0701] Output: The calculation results and invoices are sent to the relevant organizations, and the system provider receives revenue.

[0702] (Application example 1)

[0703] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0704] In conventional systems where users receive items seen in virtual spaces, games, or comics as real-world products, it was sometimes difficult to manufacture and deliver products that accurately reflected the user's customizations. Furthermore, there was a lack of means for users to check the items they customized, and insufficient collaboration with vendors, making it difficult to improve the user experience. Furthermore, improving the customization experience in virtual stores using smart devices was also a challenge.

[0705] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0706] In this invention, the server includes a means for allowing a user to select an item seen in a virtual reality space, electronic game, or comic; a means for setting customization options, such as color, size, and material, for the selected item; and a means for displaying an image generated based on the customization options. This allows the user to browse a virtual store and customize the item via a smart device. The server also includes a means for confirming the generated image and confirming order details, and a means for linking the confirmed order details to a manufacturer and having the product manufactured and delivered. This ensures that products reflecting the user's customization are manufactured and delivered. Furthermore, the server includes a means for generating an image based on the customization options using a generative AI model and a means for using prompt sentences, enabling the server to quickly and accurately generate an image that accurately reflects the user's customization, improving the user experience.

[0707] "User" refers to an individual who selects and customizes an item seen in a virtual reality space, electronic game, or comic, and then checks the generated image and places an order.

[0708] A "virtual reality space" is a virtual environment created using computer or digital technology that users can interact with.

[0709] An "electronic game" is an interactive entertainment system that is implemented using digital devices.

[0710] "Manga" is content that combines narrative and illustrations using visual storytelling.

[0711] "Items" is a general term for items that users encounter in virtual reality spaces, electronic games, and comics.

[0712] "Customization options" refer to setting items that a user can specify for the item they have selected, including color, size, material, and the like.

[0713] "Generated Image" means a visual representation generated by an AI generative model based on customization options set by a user.

[0714] "Order details" are detailed information that allows the user to check the generated image and confirm the purchase.

[0715] A "manufacturer" is a company or organization that receives a user's order, manufactures the actual goods, and delivers them.

[0716] A "smart device" is an electronic device that can be carried by a user and that can connect to the Internet and run applications, and includes smartphones, smart glasses, etc.

[0717] A "virtual store" is a virtual shopping area built in a virtual reality space where users can browse, customize, and order items.

[0718] A "generative AI model" is an artificial intelligence algorithm used to generate an image of an item based on a user's customization options.

[0719] A "prompt" is textual input that instructs a generative AI model to generate a specific image.

[0720] This invention relates to a system that allows users to select an item they see in a virtual reality space, an electronic game, or a comic, customize it, and receive it as a real product. This system allows users to customize the item to their liking and easily order and purchase it. The program processing of this system is explained in detail below.

[0721] Hardware and Software Configuration

[0722] The system uses the following hardware and software:

[0723] Smart devices: Used by users to access and customize virtual spaces. Specifically, these include smartphones and smart glasses.

[0724] Server: Receives requests from users and performs authentication, data processing, image generation, and coordination of order details.

[0725] Flask: Used as a server-side web application framework.

[0726] MongoDB: Used as a database to store user information, product information, customization information, and order information.

[0727] Generative AI Model: The AI ​​model used to generate images of customized items. Specifically, it utilizes the Gemini AI API.

[0728] System processing flow

[0729] 1. User login: The user accesses the system using a smart device and enters their username and password on the login page for authentication. The server retrieves the user information from MongoDB and performs authentication.

[0730] 2. Product category selection: After successful authentication, the user selects a product category, for example, "Sword model" from the "Game items" category.

[0731] 3. Setting customization options: The user sets customization options for the selected item, such as color, size, material, etc. For example, the color of the "model sword" can be silver, the length can be 50 cm, and the material can be wood.

[0732] 4. Image Generation: The server calls the generative AI model (Gemini AI) based on the customization information and generates an image of the customized item. The generated image is displayed to the user.

[0733] 5. Confirmation of order: The user checks the generated image and confirms the order details. The server saves the order details in the database and generates information to be shared with related vendors.

[0734] 6. Linking to vendor: The server sends the order information via the vendor's API and waits for the order confirmation from the vendor. The vendor starts production based on the received order information.

[0735] 7. Notification and Delivery: The server receives the completed product and notifies the user via email or app notification. The user can check the progress on the card authentication page.

[0736] Specific prompt examples

[0737] Here are some example prompts to input to a generative AI model:

[0738] Generate images for the following customized items:

[0739] Item: Model sword

[0740] Color: Blue

[0741] Length: 60cm

[0742] Material: Iron

[0743] In this way, this system allows users to customize items they see in a virtual space and receive them as real products. By combining smart devices with advanced AI technology, it improves the user experience and realizes a smooth ordering, manufacturing, and delivery process.

[0744] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0745] Step 1:

[0746] A user accesses the system using a smart device and enters login information (user name and password).

[0747] Input: Username, Password

[0748] Data processing: The smart device sends the user name and password to the server, which searches for the corresponding user information in MongoDB and performs authentication.

[0749] Output: Authentication result (success / failure)

[0750] Specific operation: The server checks whether the username and password match, and if successful, displays the dashboard screen to the user.

[0751] Step 2:

[0752] The user selects a product category from the dashboard screen.

[0753] Input: Selected Product Category

[0754] Data processing: The selected category information is sent from the smart device to the server, which then retrieves the corresponding product list from MongoDB.

[0755] Output: Product list

[0756] Specific operation: The server displays the retrieved product list on the user's smart device, and the user selects a specific item from the list.

[0757] Step 3:

[0758] The user sets customization options (color, size, material, etc.) for the selected item.

[0759] Input: Customization options (color, size, material)

[0760] Data processing: Customization information is sent from the smart device to the server, which temporarily stores it in MongoDB.

[0761] Output: Save result (success / failure)

[0762] Specific operation: The server checks the received customization information, and if it is in the correct format, it temporarily saves it and proceeds to the next image generation process.

[0763] Step 4:

[0764] The server calls the generative AI model based on the saved customization information and generates an image of the customized item.

[0765] Input: Customization information

[0766] Data processing: The server sends the customization information as a prompt to the generative AI model, which then generates an image based on the customization information.

[0767] Output: The generated image

[0768] Specific operation: The server generates a prompt sentence in the following format and sends it to the generative AI model.

[0769] Generate images for the following customized items:

[0770] Item: Model sword

[0771] Color: Blue

[0772] Length: 60cm

[0773] Material: Iron

[0774] The generated image data is received and displayed to the user.

[0775] Step 5:

[0776] The user checks the generated image and confirms the order details.

[0777] Input: User confirmation (order confirmation)

[0778] Data processing: The user's confirmation is sent to the server. The server receives this, officially stores the order details in MongoDB, and prepares to send them to the relevant vendors.

[0779] Output: Order confirmation notice, order information

[0780] Specific operation: When the user presses the "Confirm Order" button, the server officially saves the order details and communicates with related vendors.

[0781] Step 6:

[0782] The server uses the vendor's API to send the order information.

[0783] Input: Order Information

[0784] Data processing: The server sends the order information to the vendor's API and receives an order confirmation.

[0785] Output: Order confirmation

[0786] Specific operation: The server sends the order information to the supplier's system and stores the order confirmation from the supplier in MongoDB.

[0787] Step 7:

[0788] The server receives the completion of the product and notifies the user.

[0789] Input: Notification of completion of production from supplier

[0790] Data processing: The server stores the received manufacturing completion notification in MongoDB and notifies the user via email or app notification.

[0791] Output: User notification

[0792] Specific operation: Based on the notification from the supplier, the server sends a notification to the user that the product has been completed, and the user can check the delivery status.

[0793] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0794] This invention combines an emotion engine with a system that allows users to customize and order items they see in virtual spaces, games, and comics, and then receive them as real-world products, thereby providing a customization and ordering process that responds to the user's emotions. Below, we will explain the program processing of this system using concrete examples.

[0795] A user accesses the system

[0796] Terminal: The user accesses the system via a browser or a dedicated app.

[0797] Example: A user uses a smartphone to access the system's login page, enters their username and password, and presses the login button.

[0798] Server: Receives login information, retrieves user information from the database, and performs authentication. After successful authentication, returns the user dashboard to the terminal.

[0799] The user selects the product they want

[0800] Device: The user clicks the "Select a product" button on the dashboard and is taken to the category selection page.

[0801] Example: A user selects "game items" and then selects "sword model."

[0802] Server: Based on the selected category, retrieves a list of related products from the database and returns it to the terminal, which then displays the product list to the user.

[0803] Set customization options

[0804] Device: The user inputs customization options such as color, size, material, etc. In addition, the emotion engine analyzes the user's emotions in real time and suggests customization options based on the user's preferences and emotions.

[0805] Example: A user sets the color of a "model sword" as silver, the length as 50 cm, and the material as wood, and consults the options suggested by the emotion engine.

[0806] Server: Receives the entered customization information and stores it in a temporary database.

[0807] Image generation and confirmation using Gemini

[0808] Server: Sends an image generation request to Gemini AI based on the saved customization information.

[0809] The server receives the image data of the customized product returned by Gemini AI and returns it to the terminal.

[0810] Terminal: The generated image is displayed to the user. The emotion engine analyzes the user's reaction in real time and readjusts the image as needed.

[0811] Example: Image data sent from Gemini AI is displayed on the device, and after the user checks the image, the emotion engine makes readjustments.

[0812] Confirmation of order details

[0813] User: Check the generated image and click the "Confirm Order" button to confirm the order.

[0814] The emotion engine displays recommendations based on the user's emotions.

[0815] Server: Receives order confirmation information and stores it in the database as official order data. Generates order information for related vendors.

[0816] Cooperation with vendors

[0817] Server: Sends order information via the vendor's API or email system. Receives order confirmation from the vendor.

[0818] The supplier starts the manufacturing process of the product based on the received order information.

[0819] User Notice and Receipt

[0820] Server: Receives notification of product manufacturing completion from the supplier. Sends notification of product manufacturing completion to the user via email or app notification.

[0821] On the device: The user receives a notification and checks the delivery status of the product on the delivery status page.

[0822] User: Receives the product and presses the receipt confirmation button on the system.

[0823] Example: After receiving the product, the user presses the receipt confirmation button in the system.

[0824] Closing process and back margin payment

[0825] Server: Periodically extracts order and sales information from the database, calculates back margins, and issues invoices to related vendors.

[0826] The server periodically calculates the back margin based on the order data and sends invoices to the relevant vendors. This process allows the system provider to earn revenue.

[0827] This allows users to easily select and confirm customized products that suit their preferences, and the emotion engine can provide an even more satisfying purchasing experience.

[0828] The processing flow will be explained below.

[0829] Step 1:

[0830] Terminal: The user accesses the system via a browser or a dedicated app, goes to the login screen, enters their username and password, and presses the login button.

[0831] Server: Receives login information, retrieves user information from the database, and performs authentication. After successful authentication, returns the user dashboard to the terminal.

[0832] Step 2:

[0833] Device: The user clicks the "Select a product" button on the dashboard and is taken to the category selection page.

[0834] Terminal: The user selects a product category, for example, "game items."

[0835] Server: Based on the selected category, retrieves a list of related products from the database and returns it to the terminal, which then displays the product list to the user.

[0836] Step 3:

[0837] Terminal: The user selects a specific product from the list, such as a "model sword," and clicks the "Customize" button.

[0838] Device: A customization screen is displayed, allowing the user to input customization options such as color, size, material, etc. Furthermore, an emotion engine analyzes the user's emotions in real time and suggests customization options based on the user's preferences and emotions.

[0839] Server: Receives the entered customization information and stores it in a temporary database.

[0840] Step 4:

[0841] Server: Sends an image generation request to Gemini AI based on the saved customization information.

[0842] Server: Receives the image data of the customized product returned from Gemini AI and returns it to the terminal.

[0843] Terminal: The generated image is displayed to the user. The emotion engine analyzes the user's reaction in real time and readjusts the image as needed.

[0844] Example: Image data sent from Gemini AI is displayed on the device, and after the user checks the image, the emotion engine makes readjustments.

[0845] Step 5:

[0846] Terminal: The user checks the generated image and clicks the "Confirm Order" button to confirm the order. The emotion engine displays recommended content based on the user's emotions.

[0847] Server: Receives order confirmation information and stores it in the database as official order data. Generates order information for related vendors.

[0848] Step 6:

[0849] Server: Sends order information via the vendor's API or email system. Receives order confirmation from the vendor.

[0850] Vendor: Starts the manufacturing process of the product based on the received order information.

[0851] Step 7:

[0852] Server: Receives notification of product manufacturing completion from the supplier. Sends notification of product manufacturing completion to the user via email or app notification.

[0853] On the device: The user receives a notification and checks the delivery status of the product on the delivery status page.

[0854] Step 8:

[0855] User: Receives the product and presses the receipt confirmation button on the system.

[0856] Server: Saves the receipt confirmation information in a database.

[0857] Step 9:

[0858] Server: Periodically extracts order and sales information from the database, calculates back margins, and issues invoices to related vendors.

[0859] Example 2

[0860] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0861] Currently, when users customize and order real-world products based on items they see in virtual spaces, electronic games, or electronic comics, it is difficult to achieve high satisfaction because there is a lack of suggestions that reflect the user's emotions and preferences. Another problem is that the purchase process is hindered by the cumbersome process of readjusting the customized images after viewing them.

[0862] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0863] In this invention, the server includes means for allowing a user to select an item seen in a virtual space, electronic game, or electronic comic, means for setting customization options such as color, size, and material for the item selected by the user, means for displaying an image generated using a generative AI model based on the customization options, means for the user to check the generated image and confirm the order details, means for coordinating the confirmed order details with related vendors to manufacture and deliver the product, and means for analyzing the user's emotions in real time and suggesting customization options according to the emotions. This allows users to smoothly customize, check, and order products according to their emotions and preferences.

[0864] "User" refers to an individual or group that uses the system to customize and order items found in virtual spaces, electronic games, and electronic comics as real-world products.

[0865] "Virtual space" refers to a virtual environment that is different from the real world and is constructed on the Internet or a computer network.

[0866] An "electronic game" is a type of interactive electronic entertainment played using a computer or gaming console.

[0867] "Digital comics" are manga and graphic novels available in digital format.

[0868] "Items" refer to items or objects that users encounter in virtual spaces, electronic games, and electronic comics.

[0869] "Customization options" refers to the choices and settings such as color, size, material, etc. that a user sets for an item.

[0870] A "generative AI model" is a model that uses artificial intelligence technology to generate images and data based on customization options selected by the user.

[0871] The "emotion engine" is part of a system that analyzes users' emotions in real time and provides customization options and suggestions based on those emotions.

[0872] "Affiliated Businesses" refers to companies or corporations that receive orders confirmed by users and manufacture and deliver products.

[0873] "Order details" refers to details of the item customized by the user, and includes specific instructions and specifications for manufacturing and delivery.

[0874] "Back margin" refers to a commission or margin on sales that is paid by related parties to the system provider.

[0875] The present invention relates to a system that allows users to customize items found in virtual spaces, electronic games, and electronic comics to their liking and receive them as real-world products. This system incorporates an emotion engine that analyzes the user's emotions in real time, and provides customization options and suggestions based on the emotions. A specific implementation of the system is described below.

[0876] System configuration and hardware / software usage examples

[0877] This system mainly consists of three components: a server, a terminal, and a user. The server includes a database, a public web server, an API server, and a generative AI model. The terminal is a smartphone, tablet, or personal computer that users access. The emotion engine is a software module that performs real-time emotion analysis of user input data.

[0878] A user accesses the system

[0879] Terminal: The user accesses the system via a smartphone or PC browser, or a dedicated app. The user accesses the system's login page, enters their username and password, and presses the login button. Built-in security features ensure data safety.

[0880] Example: A user accesses "example.com / login", enters the username "user123" and password "password123", and clicks the login button.

[0881] Server: Receives login information and retrieves the corresponding user information from the database. It compares the retrieved information with the entered information, and if authentication is successful, generates the user's dashboard and returns it to the device.

[0882] Product Selection

[0883] Device: The user clicks the "Select a product" button on the dashboard to go to the category selection page, where they can select the desired product from the list of product categories offered.

[0884] Example: A user clicks the "Choose a Product" button on the dashboard and selects "Gaming Items" on the category page.

[0885] Server: Based on the selected category, retrieves a list of related products from the database and returns it to the terminal, where the user can view the product list.

[0886] Setting customization options

[0887] Terminal: The user inputs customization options such as color, size, material, etc. The emotion engine analyzes the user's emotions in real time and suggests customization options based on their preferences and emotions.

[0888] Example: A user sets the color of a "model sword" to "silver," the length to "50cm," and the material to "wood." The emotion engine analyzes the user's input and presents additional customization options.

[0889] Server: Receives the entered customization information and stores it in a temporary database.

[0890] Image generation and confirmation

[0891] Server: Based on the saved customization information, the server sends an image generation request to the generative AI model. The generative AI model generates an image that reflects the customization options and sends it back to the server.

[0892] Terminal: The generated image is displayed to the user. The emotion engine analyzes the user's reactions in real time and makes suggestions to readjust the image if necessary.

[0893] Example: A customized image sent from the generative AI model is displayed on the device, and the user can review it. The emotion engine will make adjustments as necessary.

[0894] Confirmation of order details

[0895] User: Check the generated image and if satisfied, click the "Confirm Order" button. The emotion engine may also display recommendations based on the user's emotions.

[0896] Server: Receives order confirmation information and stores it in the database as official order data. Generates order information for related vendors and communicates it to the vendors.

[0897] Cooperation with vendors

[0898] Server: Sends order information via the vendor's API or email system and receives order confirmation from the vendor.

[0899] Example: Use a vendor API to send order information and get order confirmation from the vendor.

[0900] User Notice and Receipt

[0901] Server: Receives notification of product manufacturing completion from the supplier and sends notification of product manufacturing completion to the user via email or app notification.

[0902] On the device: The user receives a notification and checks the product delivery status on the delivery status page.

[0903] User: After receiving the product, press the confirmation button on the system.

[0904] Example: After a user receives a product, they press the "Confirm Receipt" button in the system.

[0905] Closing process and back margin payment

[0906] Server: Periodically extracts order and sales information from the database, calculates back margins, and issues invoices to related vendors.

[0907] Example: The server calculates the back margin based on the order data and sends an invoice to the relevant vendor.

[0908] Prompt Sentence Examples

[0909] "Design a system that allows users to customize items using an emotion engine and experience the ordering process. Required functionality includes login, product selection, customization options, image generation using generative AI, ordering, vendor integration, notifications, receipt confirmation, and closing."

[0910] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0911] Step 1: Access the system

[0912] Terminal: The user accesses the system using a smartphone or PC browser, or a dedicated app. The user enters their username and password on the login page and presses the login button.

[0913] Input: Username "user123", Password "password123"

[0914] Output: User dashboard

[0915] Specific operation: The user accesses "example.com / login", enters the username and password, and clicks the login button.

[0916] Server: Receives login information and retrieves the corresponding user information from the database. The retrieved information is compared with the entered information, and if authentication is successful, a user dashboard is generated and returned to the terminal.

[0917] Input: Login information (username and password)

[0918] Output: Authentication results and user dashboard

[0919] Specific operation: The server retrieves user information from the database, performs authentication, and if successful, creates and returns a user dashboard.

[0920] Step 2: Select product category

[0921] On the device: The user clicks the "Select a product" button on the dashboard, which takes them to the category selection page. The user then selects the desired category from the list of product categories provided.

[0922] Input: Category selection request

[0923] Output: Product category page

[0924] Specific behavior: The user clicks the "Choose a Product" button on the dashboard and selects the "Game Items" category.

[0925] Server: Based on the selected category, retrieves a list of related products from the database and returns it to the terminal, which then displays the product list to the user.

[0926] Input: Category selection information

[0927] Output: Product list

[0928] Specific operation: The server retrieves data in the "game items" category from the database, sends a product list to the user's terminal, and displays it.

[0929] Step 3: Configure customization options

[0930] Terminal: The user inputs customization options such as color, size, material, etc. The emotion engine analyzes the user's emotions in real time and suggests customization options based on the user's preferences and emotions.

[0931] Input: Customization option information

[0932] Output: Optimized customization options

[0933] What it does: The user sets the color of the "model sword" to "silver," the length to "50cm," and the material to "wood." The emotion engine uses these inputs to suggest additional customization options.

[0934] Server: Receives the entered customization information and stores it in a temporary database.

[0935] Input: User-entered customization information

[0936] Output: Save results to a temporary database

[0937] Specific operation: The server receives the customization information of "silver, 50cm, wood" and stores it in a temporary database.

[0938] Step 4: Image generation and verification

[0939] Server: Based on the saved customization information, the server sends an image generation request to the generative AI model. The generative AI model generates an image that reflects the customization options and sends it back to the server.

[0940] Input: Customization information

[0941] Output: Generated customized image

[0942] Specific operation: The server requests the generative AI model to generate an image of a "silver, 50 cm, wooden sword" and receives the generated image data.

[0943] Terminal: The generated image is displayed to the user. The emotion engine analyzes the user's reactions in real time and makes suggestions to readjust the image if necessary.

[0944] Input: Customized image data

[0945] Output: Optimized customized image

[0946] Specific operation: The image sent from the generative AI model is displayed on the device, and the emotion engine suggests readjustments based on the user's emotional response.

[0947] Step 5: Confirm your order

[0948] User: Check the generated image and if satisfied, click the "Confirm Order" button. The emotion engine may also display recommendations based on the user's emotions.

[0949] Input: Order confirmation request

[0950] Output: Order confirmation notification

[0951] Specific operation: The user checks the image and clicks the "Confirm Order" button.

[0952] Server: Receives order confirmation information and stores it in the database as official order data. Generates order information for related vendors and communicates it to the vendors.

[0953] Input: Order confirmation information

[0954] Output: Official order data and order information to vendors

[0955] Specific operation: The server saves the order information in a database and prepares to send the order information to the supplier.

[0956] Step 6: Contact a vendor

[0957] Server: Sends order information via the vendor's API or email system and receives order confirmation from the vendor.

[0958] Input: Order data

[0959] Output: Vendor order confirmation

[0960] Specific operation: The server sends the order information to the supplier and receives an order confirmation.

[0961] Step 7: User Notification and Acceptance

[0962] Server: Receives notification from the supplier that product manufacturing is complete and sends notification of product manufacturing completion to the user via email or app notification.

[0963] Input: Product manufacturing completion notification

[0964] Output: Product manufacturing completion notification to user

[0965] Specific operation: The server receives the product manufacturing completion information and sends a notification to the user.

[0966] On the device: The user receives a notification and checks the product delivery status on the delivery status page.

[0967] Input: Receive completion notification

[0968] Output: Display delivery status

[0969] What happens: A user checks the delivery status of a product online.

[0970] User: After receiving the product, press the confirmation button on the system.

[0971] Input: Receipt Acknowledgment Request

[0972] Output: Notification of receipt

[0973] Specific operation: The user receives the product and clicks the "Confirm receipt" button.

[0974] Step 8: Closing and margin payment

[0975] Server: Periodically extracts order and sales information from the database, calculates back margins, and issues invoices to related vendors.

[0976] Input: Order and sales data

[0977] Output: Back margin calculation results and invoice

[0978] Specific operation: The server calculates the back margin based on the order data for the specified period and sends an invoice to the relevant vendor.

[0979] (Application example 2)

[0980] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0981] In current systems that allow users to receive items seen in virtual spaces or games as physical goods in the real world, the customization experience is complicated, making it difficult to achieve highly satisfying customization that reflects the user's emotions.Another issue is that the wide range of customization options makes it difficult for users to make the optimal choice.

[0982] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0983] In this invention, the server includes: a means for allowing a user to select an item seen in a virtual space, game, or comic; a means for setting customization options such as color, size, and material for the item selected by the user; an emotion analysis means for analyzing the user's emotions in real time and suggesting customization options; a means for displaying an image generated based on the customization options; a means for the user to check the generated image and confirm the order details; and a means for coordinating the confirmed order details with related vendors to have the product manufactured and delivered. This enables a highly satisfying customization and ordering process that reflects the user's emotions.

[0984] A "virtual space" is a virtual environment generated by a computer.

[0985] A "game" is an interactive activity under prescribed rules for recreational or competitive purposes.

[0986] "Comics" is a form of publication that tells a story through pictures and text.

[0987] An "item" is a thing or element that exists in relation to a particular use or purpose.

[0988] "Customization options" are options that allow users to set the color, size, material, etc. of an item according to their preferences.

[0989] "Emotion analysis means" is a technology that analyzes emotions in real time from a user's facial expressions and behavior.

[0990] A "generated image" is a visual representation generated based on the customization options set by the user.

[0991] "Order details" refers to product specifications and order information confirmed by the user.

[0992] "Affiliated businesses" are companies or organizations that manufacture and deliver products based on user orders.

[0993] A "generative AI model" is an algorithm that uses artificial intelligence to generate images or text from data.

[0994] A "system" is a set of processes or devices consisting of multiple elements configured to achieve a specific purpose.

[0995] This invention provides a customization and ordering process that responds to the user's emotions by combining an emotion analysis engine with a system that allows users to customize items they see in virtual spaces, games, and comics and receive them as real products. The following describes an embodiment of the invention.

[0996] Hardware and software used

[0997] 1. Smartphone: Users use iOS or Android devices, and a dedicated application is installed on the smartphone.

[0998] 2. Emotion Analysis Engine: Affectiva SDK is used. This SDK analyzes the user's facial expression data and grasps their emotional state in real time.

[0999] 3. Generative AI model: OpenAI GPT and Gemini AI are used. OpenAI GPT performs natural language analysis and generation, and Gemini AI generates images based on customization options.

[1000] 4. Server: The server uses AWS (Amazon Web Services), which handles back-end processing, data storage, and vendor integration.

[1001] 5. Database: MySQL and DynamoDB are used to store customization information, order information, user information, etc.

[1002] 6. Notification system: Use Firebase Cloud Messaging (FCM) to send notifications to users.

[1003] 7. API communication: Data communication between systems is carried out using RESTful APIs.

[1004] Specific examples of the invention

[1005] A user accesses the system

[1006] Users access the system from a smartphone application and log in by entering their username and password. The server receives the login information, retrieves user information from the database, and performs authentication. If authentication is successful, it returns the user dashboard to the device.

[1007] The user selects the product they want

[1008] The user clicks the "Select a Product" button on the smartphone dashboard, which takes them to a category selection page. The user selects "Game Items" and then "Model Sword" from the list. The server retrieves a list of related products from the database based on the selected category and returns it to the device. The user then checks the product list on their smartphone.

[1009] Set customization options

[1010] Users input customization options such as color, size, and material. At this time, the sentiment analysis engine analyzes the user's emotions in real time and suggests customization options based on the user's preferences and emotions. For example, if a user sets the color of a "model sword" as silver, the length as 50 cm, and the material as wood, the sentiment analysis engine will suggest better options.

[1011] Image generation and confirmation using Gemini

[1012] The server sends an image generation request to Gemini AI, a generative AI model, based on the saved customization information. The image data of the customized product returned by Gemini AI is then received and returned to the device. The user can then view the generated image on their smartphone, and the sentiment analysis engine analyzes the user's reactions in real time, readjusting the image as necessary.

[1013] Confirmation of order details

[1014] The user checks the generated image and clicks the "Confirm Order" button to confirm the order. At this time, the sentiment analysis engine displays recommendations based on the user's emotions. The server receives the order confirmation information and stores it in the database as official order data. It also generates order information for related vendors.

[1015] Cooperation with vendors

[1016] The server sends the order information via the supplier's API or email system, receives an order confirmation from the supplier, and the supplier starts the manufacturing process based on the order information.

[1017] User Notice and Receipt

[1018] When the server receives a notification from the supplier that the product has been completed, it will notify the user by email or app notification. The user will receive the notification and check the product's delivery status on the delivery status page. When the user receives the product, they can press the receipt confirmation button on the system to complete the receipt.

[1019] Closing process and back margin payment

[1020] The server periodically extracts order and sales information from the database, calculates the back margin, and issues invoices to related vendors, allowing the system provider to earn revenue.

[1021] Prompt Sentence Examples

[1022] Below is an example prompt that can be input to a generative AI model to generate suggested customization options based on sentiment analysis.

[1023] text

[1024] The user selected the "Sword Model." Based on the user's facial expression data, their current emotional state is "Happy." Suggest the best customization options for this sword model based on the emotion of "Happy." Recommend color, material, and design changes as needed.

[1025] This prompt provides emotional data to the generative AI model, which then suggests appropriate customization options. This approach allows users to customize products in a way that best suits their emotions, resulting in a satisfying shopping experience.

[1026] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1027] Step 1:

[1028] A user accesses the system and enters their login information.

[1029] Input: Username and Password

[1030] Specific operation: The user accesses the login screen of the smartphone app, enters their username and password, and presses the login button.

[1031] Data processing: The server receives the entered login information and retrieves user information from the database (MySQL).

[1032] Output: If authentication is successful, returns the user dashboard to the device.

[1033] Step 2:

[1034] The user selects the product they want.

[1035] Input: "Select a product" button and category selection (e.g., game items, sword model)

[1036] Specific Action: The user clicks the "Select a Product" button on the dashboard, goes to the category selection page, and then selects a specific item (e.g., "Model Sword") from the selected category.

[1037] Data processing: The server retrieves a list of related products from the database (DynamoDB) based on the selected category.

[1038] Output: The terminal displays the product list to the user.

[1039] Step 3:

[1040] Enter the customization options.

[1041] Input: Customization information such as item color, size, material, etc.

[1042] How it works: Users input customization options for the selected item, such as color, size, material, etc. At the same time, the emotion analysis engine (Affectiva SDK) analyzes the user's facial expression data in real time.

[1043] Data processing: The server sends the sentiment analysis data to OpenAI GPT, which then suggests customization options based on the user's emotions.

[1044] Output: Prints the suggested customization options to the terminal.

[1045] Step 4:

[1046] Check the generated image.

[1047] Input: Customization information and user emotion data

[1048] Specific operation: The user confirms the settings based on the customization options. After the settings are confirmed, the server sends an image generation request to Gemini AI based on the settings.

[1049] Data processing: Gemini AI creates a generated image based on the customization information and sends it back to the server.

[1050] Output: The server returns the generated image to the device, which displays it to the user. The sentiment analysis engine then analyzes the user's reaction again and adjusts the image if necessary.

[1051] Step 5:

[1052] Confirm the order details.

[1053] Input: Confirmed customizations and generated images

[1054] Specific operation: The user checks the generated image and clicks the "Confirm Order" button. At this time, the sentiment analysis engine displays recommendations based on the user's emotions.

[1055] Data processing: The server receives the order confirmation information, stores it in the database as official order data, and generates order information for related vendors.

[1056] Output: The order information is sent to the relevant supplier and the manufacturing process is initiated.

[1057] Step 6:

[1058] Product manufacturing and user notification.

[1059] Input: Production completion notice from supplier

[1060] Specific operation: The server receives notification from the relevant supplier that the product has been completed. Based on this information, it uses Firebase Cloud Messaging (FCM) to notify the user that the product has been completed.

[1061] Data processing: Generates notification information and sends it to the user's device.

[1062] Output: The user receives a notification on their smartphone that production is complete and checks the delivery status on the delivery status page.

[1063] Step 7:

[1064] Receipt and confirmation in the system.

[1065] Input: When the user receives the product, they press the "Confirm Receipt" button.

[1066] Specific operation: After receiving the product, the user presses the "Confirm receipt" button on the smartphone app, which causes the system to confirm receipt.

[1067] Data processing: The server records the received data and stores it in a database.

[1068] Output: The receipt confirmation information is saved in the database and the product receipt procedure is completed.

[1069] Step 8:

[1070] Regular data processing and back margin calculation.

[1071] Input: Order and sales information in the database

[1072] Specific operation: The server periodically extracts order information and sales information from the database, calculates the back margin based on this information, and issues invoices to related vendors.

[1073] Data processing: Analyze and calculate sales data and order information.

[1074] Output: Invoices are sent to the relevant vendors and back margin payments are processed.

[1075] As described above, by clearly indicating the specific operations, inputs, and outputs at each step, the user can smoothly complete the customization and ordering process and achieve a high level of satisfaction.

[1076] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1077] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1078] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[1079] [Third embodiment]

[1080] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[1081] 5, the data processing system 310 includes the data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[1082] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1083] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[1084] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1085] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1086] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1087] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1088] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1089] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1090] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1091] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[1092] This invention relates to a system that allows users to customize and order items they see in virtual spaces, games, and comics, and then receive them as real-world products. This system allows users to freely customize specific items to their liking and easily order and purchase them. The program processing of this system is explained below with specific examples.

[1093] A user accesses the system

[1094] Terminal: The user accesses the system via a browser or a dedicated app.

[1095] Example: A user uses a smartphone to access the system's login page, enters their username and password, and presses the login button.

[1096] Server: Receives login information and performs authentication.

[1097] The server retrieves user information from the database, verifies the entered username and password, and if authentication is successful, returns the dashboard screen to the terminal.

[1098] The user selects the product they want

[1099] Terminal: User selects product category and chooses specific items.

[1100] Example: A user selects "game items" and then selects "sword model."

[1101] Server: Serves a list of products based on the selected category.

[1102] The server retrieves products related to the "game item" from the database and returns the list to the terminal, which then displays the product list to the user.

[1103] Set customization options

[1104] Terminal: The user sets the product's color, size, material, etc.

[1105] Example: A user sets the color of a "sword model" to silver, the length to 50 cm, and the material to wood.

[1106] Server: Receives and temporarily saves the configured customization options.

[1107] The server temporarily stores the received customization information in a database.

[1108] Image generation and confirmation using Gemini

[1109] Server: Sends customization information to Gemini AI and generates images.

[1110] The server calls the Gemini API based on the stored customization information to generate an image of the customized item.

[1111] Terminal: displays the generated image to the user.

[1112] Example: Image data sent from Gemini AI is displayed on the device, and the user checks the image.

[1113] Confirmation of order details

[1114] User: Check the generated image and confirm the order details.

[1115] If the user is satisfied with the image, he / she presses the "Confirm Order" button to confirm the order.

[1116] Server: Stores the order details in a database and generates information to be shared with related vendors.

[1117] The server officially stores the order details in a database and transmits the order information to related vendors.

[1118] Cooperation with vendors

[1119] Server: Sends order information to related vendors.

[1120] The server sends the order information via the vendor's API and waits for order confirmation from the vendor.

[1121] Supplier: Starts production based on the order information.

[1122] Example: A vendor receives an order and manufactures a model sword, preparing it for delivery.

[1123] User Notice and Receipt

[1124] Server: Notify the user that the product is complete.

[1125] The server receives a notification of production completion from the supplier and notifies the user via email or app notification.

[1126] Terminal: User receives notification and checks delivery status.

[1127] Users receive notifications and view progress on the delivery status page.

[1128] User: Receives the product and confirms receipt.

[1129] Example: After receiving the product, the user presses the receipt confirmation button on the system.

[1130] Closing process and back margin payment

[1131] Server: Calculates back margins, closes the transaction, and bills the sales company.

[1132] The server periodically extracts order and sales information from the database, calculates the back margin based on the contract with the related vendor, and issues invoices, which allows the system provider to earn revenue.

[1133] The processing flow will be explained below.

[1134] Step 1:

[1135] Terminal: The user accesses the system and moves to the login screen. They enter their username and password and press the login button.

[1136] Server: Receives login information, retrieves user information from the database, and performs authentication. After successful authentication, returns the user dashboard to the terminal.

[1137] Step 2:

[1138] Device: User clicks the "Choose a Product" button from the dashboard. The category selection page appears.

[1139] Device: The user selects a product category, for example, "Game Items."

[1140] Server: Based on the selected category, retrieves a list of related products from the database and returns it to the terminal, which then displays the product list to the user.

[1141] Step 3:

[1142] Terminal: The user selects a specific product from the list, such as a "model sword," and clicks the "Customize" button.

[1143] Device: A customization screen appears, allowing the user to enter customization options such as color, size, and material.

[1144] Server: Receives the entered customization information and stores it in a temporary database.

[1145] Step 4:

[1146] Server: Sends an image generation request to Gemini AI based on the saved customization information.

[1147] Server: Receives the image data of the customized product returned from Gemini AI and returns it to the terminal.

[1148] Terminal: Displays customized product images to the user.

[1149] Step 5:

[1150] Terminal: The user checks the generated image and clicks the "Confirm Order" button to confirm the order.

[1151] Server: Receives order confirmation information and stores it in the database as official order data. Generates order information for related vendors.

[1152] Step 6:

[1153] Server: Sends order information via the vendor's API or email system. Receives order confirmation from the vendor.

[1154] Vendor: Starts the manufacturing process of the product based on the received order information.

[1155] Step 7:

[1156] Server: Receives notification of product manufacturing completion from the supplier. Sends notification of product manufacturing completion to the user via email or app notification.

[1157] On the device: The user receives a notification and checks the delivery status of the product on the delivery status page.

[1158] Step 8:

[1159] User: Receives the product and presses the receipt confirmation button on the system.

[1160] Server: Saves the receipt confirmation information in a database.

[1161] Step 9:

[1162] Server: Periodically extracts order and sales information from the database, calculates back margins, and issues invoices to related vendors.

[1163] Example 1

[1164] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1165] Conventional online shopping systems make it difficult for users to customize and order digital items as real physical products. Furthermore, it is difficult for users to visually confirm the customized items, and there are many cases where the product does not meet expectations after ordering. Furthermore, there are problems with efficiently calculating back margins and billing procedures linked to the product manufacturing process.

[1166] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1167] In this invention, the server includes a means for allowing a user to select a digital item to be seen in a virtual space or entertainment content, a means for setting customization options such as color, size, and material for the selected digital item, and a means for displaying an image generated using a generative AI model based on the customization options. This allows a user to easily customize a digital item while visually checking it and then order it as a real physical product. Furthermore, by linking with the manufacturing process, back margin calculations and billing procedures can be efficiently performed.

[1168] "User" means an individual or entity that utilizes the System to select, customize, and order Digital Items as physical products.

[1169] "Digital items" refer to various products and objects that are visually recognized in virtual spaces and entertainment content.

[1170] "Customization options" are options that allow a user to change the color, size, material, etc. of a digital item selected by the user according to the user's preferences.

[1171] A "generative AI model" is an artificial intelligence algorithm for generating images of a digital item based on customization options.

[1172] "Image generation" is the process of using a generative AI model to create a visual display based on customization options.

[1173] A "manufacturing organization" is a company or factory that receives orders from users, manufactures physical products, and delivers them.

[1174] A "physical product" is a physical product that is created by physically creating a digital item.

[1175] "Back margin" refers to the commission or profit sharing that a system provider receives from a manufacturing organization based on a contract with the manufacturing organization for orders or sales.

[1176] "Notification means" refers to the method by which the system keeps users informed about product completion and delivery status.

[1177] A "prompt sentence" is text data input into a generative AI model, and contains customization information for image generation.

[1178] This invention relates to a system that allows users to select digital items that appear in virtual spaces or entertainment content, customize them, and order them as physical products. This system realizes a smooth process from customization to ordering and product delivery through mutual cooperation between the user, terminal, and server.

[1179] First, users access the system via a smartphone or computer browser or a dedicated app. They access the system's login page and log in by entering their username and password. After that, the dashboard screen is displayed, allowing the user to freely operate the system.

[1180] Next, the user selects a product category from the displayed dashboard and then selects a specific digital item from that category. For example, the user can select "Game Items" and then select a "Sword Model." For the selected digital item, the user can set customization options such as color, size, and material.

[1181] Once customization is complete, the server uses a generative AI model such as Gemini to generate an image based on the customization options. This generated image is then displayed to the user via their device. The user can review the displayed image and customize it again if necessary.

[1182] If the user is satisfied with the generated image, they press the "Confirm Order" button to confirm the order. The server officially saves this order in the database and shares the information with the manufacturing organization. At this point, the manufacturing organization begins manufacturing the physical product based on the received customization information.

[1183] Once the product is completed, the manufacturing organization sends the completion information to the server, and the server notifies the user. The user receives the notification and can check the delivery status. Once the product is delivered to the user, the user confirms receipt in the system and the process is complete. Finally, the server calculates the back margin based on the order information and bills the relevant organization.

[1184] For example, suppose a user selects a "model sword," sets the color to silver, the length to 50cm, and the material to wood, and requests that an image be generated. An example prompt for this would be:

[1185] Example prompt sentence:

[1186] "The user has selected a model sword and specified customization options: color is silver, length is 50cm, and material is wood. Use this information to generate an image of the customized model sword."

[1187] As a result, the system of the present invention provides an efficient and intuitive platform that allows users to easily customize digital items visually and order them as real physical products, while also realizing monetization for the system provider by automating the manufacturing process and back-margin calculation.

[1188] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1189] Step 1: User Login

[1190] Input: Username and Password

[1191] Specific operation: The user accesses the system's login page using a browser on their device (smartphone or computer) or a dedicated app, enters their username and password, and presses the login button.

[1192] Data processing and calculation: The server receives the entered username and password, retrieves the user information from the database, and performs verification.

[1193] Output: If the match is successful, the dashboard screen is returned to the terminal, allowing the user to operate the system.

[1194] Step 2: Select your product

[1195] Input: User selection of product category and specific item

[1196] Specific operation: The user selects a product category (e.g., "game items") from the dashboard and then selects a specific item (e.g., "sword model") from that category.

[1197] Data processing and calculation: The server receives the user's selection information and retrieves the corresponding product list from the database.

[1198] Output: The server returns the product list to the terminal, which displays the list to the user.

[1199] Step 3: Configure customization options

[1200] Input: Customization options such as color, size, material, etc. set by the user.

[1201] Specific operation: The user sets the color of the selected "sword model" to silver, the length to 50cm, and the material to wood.

[1202] Data processing and calculation: The server receives the customization options and temporarily stores them in the database.

[1203] Output: The customization information is saved in the database.

[1204] Step 4: Generate customized images

[1205] Input: Customization option information

[1206] Specific operation: The server calls the API of a generative AI model (e.g., Gemini) based on the saved customization information to generate an image of the customized item. It sends the following prompt: "The user selected a sword model and specified customization options. The color is silver, the length is 50 cm, and the material is wood. Please generate an image of the customized sword model based on this information."

[1207] Data processing and calculation: The server generates an image using the generative AI model and receives the image data.

[1208] Output: The generated image data is sent from the server to the terminal and displayed.

[1209] Step 5: Confirm your order

[1210] Input: User's order confirmation instructions

[1211] Specific operation: The user checks the generated image and presses the "Confirm Order" button if satisfied.

[1212] Data processing and calculation: The server officially stores the order details in the database and generates information linked to the manufacturing organization.

[1213] Output: The purchase order is sent to the manufacturing organization and production of the product begins.

[1214] Step 6: Product Completion Notification and Delivery Confirmation

[1215] Input: Product completion notification from manufacturing organization

[1216] Specific operation: The server receives a product completion notification from the manufacturing organization and notifies the user.

[1217] Data processing and calculation: The server sends the product completion information to the user via email or app notification.

[1218] Output: User receives notification and checks delivery status.

[1219] Step 7: Acknowledgement

[1220] Input: User's receipt confirmation instructions

[1221] Specific operation: After receiving the product, the user presses the receipt confirmation button on the system.

[1222] Data processing and calculation: The server saves the receipt confirmation data in the database.

[1223] Output: The receipt confirmation information is saved in the system.

[1224] Step 8: Calculate back margin

[1225] Input: Order and sales information

[1226] Specific operation: The server periodically extracts order and sales information from the database.

[1227] Data processing and calculation: The server calculates the back margin based on the order and sales information and issues an invoice.

[1228] Output: The calculation results and invoices are sent to the relevant organizations, and the system provider receives revenue.

[1229] (Application example 1)

[1230] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1231] In conventional systems where users receive items seen in virtual spaces, games, or comics as real-world products, it was sometimes difficult to manufacture and deliver products that accurately reflected the user's customizations. Furthermore, there was a lack of means for users to check the items they customized, and insufficient collaboration with vendors, making it difficult to improve the user experience. Furthermore, improving the customization experience in virtual stores using smart devices was also a challenge.

[1232] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1233] In this invention, the server includes a means for allowing a user to select an item seen in a virtual reality space, electronic game, or comic; a means for setting customization options, such as color, size, and material, for the selected item; and a means for displaying an image generated based on the customization options. This allows the user to browse a virtual store and customize the item via a smart device. The server also includes a means for confirming the generated image and confirming order details, and a means for linking the confirmed order details to a manufacturer and having the product manufactured and delivered. This ensures that products reflecting the user's customization are manufactured and delivered. Furthermore, the server includes a means for generating an image based on the customization options using a generative AI model and a means for using prompt sentences, enabling the server to quickly and accurately generate an image that accurately reflects the user's customization, improving the user experience.

[1234] "User" refers to an individual who selects and customizes an item seen in a virtual reality space, electronic game, or comic, and then checks the generated image and places an order.

[1235] A "virtual reality space" is a virtual environment created using computer or digital technology that users can interact with.

[1236] An "electronic game" is an interactive entertainment system that is implemented using digital devices.

[1237] "Manga" is content that combines narrative and illustrations using visual storytelling.

[1238] "Items" is a general term for items that users encounter in virtual reality spaces, electronic games, and comics.

[1239] "Customization options" refer to setting items that a user can specify for the item they have selected, including color, size, material, and the like.

[1240] "Generated Image" means a visual representation generated by an AI generative model based on customization options set by a user.

[1241] "Order details" are detailed information that allows the user to check the generated image and confirm the purchase.

[1242] A "manufacturer" is a company or organization that receives a user's order, manufactures the actual goods, and delivers them.

[1243] A "smart device" is an electronic device that can be carried by a user and that can connect to the Internet and run applications, and includes smartphones, smart glasses, etc.

[1244] A "virtual store" is a virtual shopping area built in a virtual reality space where users can browse, customize, and order items.

[1245] A "generative AI model" is an artificial intelligence algorithm used to generate an image of an item based on a user's customization options.

[1246] A "prompt" is textual input that instructs a generative AI model to generate a specific image.

[1247] This invention relates to a system that allows users to select an item they see in a virtual reality space, an electronic game, or a comic, customize it, and receive it as a real product. This system allows users to customize the item to their liking and easily order and purchase it. The program processing of this system is explained in detail below.

[1248] Hardware and Software Configuration

[1249] The system uses the following hardware and software:

[1250] Smart devices: Used by users to access and customize virtual spaces. Specifically, these include smartphones and smart glasses.

[1251] Server: Receives requests from users and performs authentication, data processing, image generation, and coordination of order details.

[1252] Flask: Used as a server-side web application framework.

[1253] MongoDB: Used as a database to store user information, product information, customization information, and order information.

[1254] Generative AI Model: The AI ​​model used to generate images of customized items. Specifically, it utilizes the Gemini AI API.

[1255] System processing flow

[1256] 1. User login: The user accesses the system using a smart device and enters their username and password on the login page for authentication. The server retrieves the user information from MongoDB and performs authentication.

[1257] 2. Product category selection: After successful authentication, the user selects a product category, for example, "Sword model" from the "Game items" category.

[1258] 3. Setting customization options: The user sets customization options for the selected item, such as color, size, material, etc. For example, the color of the "model sword" can be silver, the length can be 50 cm, and the material can be wood.

[1259] 4. Image Generation: The server calls the generative AI model (Gemini AI) based on the customization information and generates an image of the customized item. The generated image is displayed to the user.

[1260] 5. Confirmation of order: The user checks the generated image and confirms the order details. The server saves the order details in the database and generates information to be shared with related vendors.

[1261] 6. Linking to vendor: The server sends the order information via the vendor's API and waits for the order confirmation from the vendor. The vendor starts production based on the received order information.

[1262] 7. Notification and Delivery: The server receives the completed product and notifies the user via email or app notification. The user can check the progress on the card authentication page.

[1263] Specific prompt examples

[1264] Here are some example prompts to input to a generative AI model:

[1265] Generate images for the following customized items:

[1266] Item: Model sword

[1267] Color: Blue

[1268] Length: 60cm

[1269] Material: Iron

[1270] In this way, this system allows users to customize items they see in a virtual space and receive them as real products. By combining smart devices with advanced AI technology, it improves the user experience and realizes a smooth ordering, manufacturing, and delivery process.

[1271] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1272] Step 1:

[1273] A user accesses the system using a smart device and enters login information (user name and password).

[1274] Input: Username, Password

[1275] Data processing: The smart device sends the user name and password to the server, which searches for the corresponding user information in MongoDB and performs authentication.

[1276] Output: Authentication result (success / failure)

[1277] Specific operation: The server checks whether the username and password match, and if successful, displays the dashboard screen to the user.

[1278] Step 2:

[1279] The user selects a product category from the dashboard screen.

[1280] Input: Selected Product Category

[1281] Data processing: The selected category information is sent from the smart device to the server, which then retrieves the corresponding product list from MongoDB.

[1282] Output: Product list

[1283] Specific operation: The server displays the retrieved product list on the user's smart device, and the user selects a specific item from the list.

[1284] Step 3:

[1285] The user sets customization options (color, size, material, etc.) for the selected item.

[1286] Input: Customization options (color, size, material)

[1287] Data processing: Customization information is sent from the smart device to the server, which temporarily stores it in MongoDB.

[1288] Output: Save result (success / failure)

[1289] Specific operation: The server checks the received customization information, and if it is in the correct format, it temporarily saves it and proceeds to the next image generation process.

[1290] Step 4:

[1291] The server calls the generative AI model based on the saved customization information and generates an image of the customized item.

[1292] Input: Customization information

[1293] Data processing: The server sends the customization information as a prompt to the generative AI model, which then generates an image based on the customization information.

[1294] Output: The generated image

[1295] Specific operation: The server generates a prompt sentence in the following format and sends it to the generative AI model.

[1296] Generate images for the following customized items:

[1297] Item: Model sword

[1298] Color: Blue

[1299] Length: 60cm

[1300] Material: Iron

[1301] The generated image data is received and displayed to the user.

[1302] Step 5:

[1303] The user checks the generated image and confirms the order details.

[1304] Input: User confirmation (order confirmation)

[1305] Data processing: The user's confirmation is sent to the server. The server receives this, officially stores the order details in MongoDB, and prepares to send them to the relevant vendors.

[1306] Output: Order confirmation notice, order information

[1307] Specific operation: When the user presses the "Confirm Order" button, the server officially saves the order details and communicates with related vendors.

[1308] Step 6:

[1309] The server uses the vendor's API to send the order information.

[1310] Input: Order Information

[1311] Data processing: The server sends the order information to the vendor's API and receives an order confirmation.

[1312] Output: Order confirmation

[1313] Specific operation: The server sends the order information to the supplier's system and stores the order confirmation from the supplier in MongoDB.

[1314] Step 7:

[1315] The server receives the completion of the product and notifies the user.

[1316] Input: Notification of completion of production from supplier

[1317] Data processing: The server stores the received manufacturing completion notification in MongoDB and notifies the user via email or app notification.

[1318] Output: User notification

[1319] Specific operation: Based on the notification from the supplier, the server sends a notification to the user that the product has been completed, and the user can check the delivery status.

[1320] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1321] This invention combines an emotion engine with a system that allows users to customize and order items they see in virtual spaces, games, and comics, and then receive them as real-world products, thereby providing a customization and ordering process that responds to the user's emotions. Below, we will explain the program processing of this system using concrete examples.

[1322] A user accesses the system

[1323] Terminal: The user accesses the system via a browser or a dedicated app.

[1324] Example: A user uses a smartphone to access the system's login page, enters their username and password, and presses the login button.

[1325] Server: Receives login information, retrieves user information from the database, and performs authentication. After successful authentication, returns the user dashboard to the terminal.

[1326] The user selects the product they want

[1327] Device: The user clicks the "Select a product" button on the dashboard and is taken to the category selection page.

[1328] Example: A user selects "game items" and then selects "sword model."

[1329] Server: Based on the selected category, retrieves a list of related products from the database and returns it to the terminal, which then displays the product list to the user.

[1330] Set customization options

[1331] Device: The user inputs customization options such as color, size, material, etc. In addition, the emotion engine analyzes the user's emotions in real time and suggests customization options based on the user's preferences and emotions.

[1332] Example: A user sets the color of a "model sword" as silver, the length as 50 cm, and the material as wood, and consults the options suggested by the emotion engine.

[1333] Server: Receives the entered customization information and stores it in a temporary database.

[1334] Image generation and confirmation using Gemini

[1335] Server: Sends an image generation request to Gemini AI based on the saved customization information.

[1336] The server receives the image data of the customized product returned by Gemini AI and returns it to the terminal.

[1337] Terminal: The generated image is displayed to the user. The emotion engine analyzes the user's reaction in real time and readjusts the image as needed.

[1338] Example: Image data sent from Gemini AI is displayed on the device, and after the user checks the image, the emotion engine makes readjustments.

[1339] Confirmation of order details

[1340] User: Check the generated image and click the "Confirm Order" button to confirm the order.

[1341] The emotion engine displays recommendations based on the user's emotions.

[1342] Server: Receives order confirmation information and stores it in the database as official order data. Generates order information for related vendors.

[1343] Cooperation with vendors

[1344] Server: Sends order information via the vendor's API or email system. Receives order confirmation from the vendor.

[1345] The supplier starts the manufacturing process of the product based on the received order information.

[1346] User Notice and Receipt

[1347] Server: Receives notification of product manufacturing completion from the supplier. Sends notification of product manufacturing completion to the user via email or app notification.

[1348] On the device: The user receives a notification and checks the delivery status of the product on the delivery status page.

[1349] User: Receives the product and presses the receipt confirmation button on the system.

[1350] Example: After receiving the product, the user presses the receipt confirmation button in the system.

[1351] Closing process and back margin payment

[1352] Server: Periodically extracts order and sales information from the database, calculates back margins, and issues invoices to related vendors.

[1353] The server periodically calculates the back margin based on the order data and sends invoices to the relevant vendors. This process allows the system provider to earn revenue.

[1354] This allows users to easily select and confirm customized products that suit their preferences, and the emotion engine can provide an even more satisfying purchasing experience.

[1355] The processing flow will be explained below.

[1356] Step 1:

[1357] Terminal: The user accesses the system via a browser or a dedicated app, goes to the login screen, enters their username and password, and presses the login button.

[1358] Server: Receives login information, retrieves user information from the database, and performs authentication. After successful authentication, returns the user dashboard to the terminal.

[1359] Step 2:

[1360] Device: The user clicks the "Select a product" button on the dashboard and is taken to the category selection page.

[1361] Terminal: The user selects a product category, for example, "game items."

[1362] Server: Based on the selected category, retrieves a list of related products from the database and returns it to the terminal, which then displays the product list to the user.

[1363] Step 3:

[1364] Terminal: The user selects a specific product from the list, such as a "model sword," and clicks the "Customize" button.

[1365] Device: A customization screen is displayed, allowing the user to input customization options such as color, size, material, etc. Furthermore, an emotion engine analyzes the user's emotions in real time and suggests customization options based on the user's preferences and emotions.

[1366] Server: Receives the entered customization information and stores it in a temporary database.

[1367] Step 4:

[1368] Server: Sends an image generation request to Gemini AI based on the saved customization information.

[1369] Server: Receives the image data of the customized product returned from Gemini AI and returns it to the terminal.

[1370] Terminal: The generated image is displayed to the user. The emotion engine analyzes the user's reaction in real time and readjusts the image as needed.

[1371] Example: Image data sent from Gemini AI is displayed on the device, and after the user checks the image, the emotion engine makes readjustments.

[1372] Step 5:

[1373] Terminal: The user checks the generated image and clicks the "Confirm Order" button to confirm the order. The emotion engine displays recommended content based on the user's emotions.

[1374] Server: Receives order confirmation information and stores it in the database as official order data. Generates order information for related vendors.

[1375] Step 6:

[1376] Server: Sends order information via the vendor's API or email system. Receives order confirmation from the vendor.

[1377] Vendor: Starts the manufacturing process of the product based on the received order information.

[1378] Step 7:

[1379] Server: Receives notification of product manufacturing completion from the supplier. Sends notification of product manufacturing completion to the user via email or app notification.

[1380] On the device: The user receives a notification and checks the delivery status of the product on the delivery status page.

[1381] Step 8:

[1382] User: Receives the product and presses the receipt confirmation button on the system.

[1383] Server: Saves the receipt confirmation information in a database.

[1384] Step 9:

[1385] Server: Periodically extracts order and sales information from the database, calculates back margins, and issues invoices to related vendors.

[1386] Example 2

[1387] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1388] Currently, when users customize and order real-world products based on items they see in virtual spaces, electronic games, or electronic comics, it is difficult to achieve high satisfaction because there is a lack of suggestions that reflect the user's emotions and preferences. Another problem is that the purchase process is hindered by the cumbersome process of readjusting the customized images after viewing them.

[1389] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1390] In this invention, the server includes means for allowing a user to select an item seen in a virtual space, electronic game, or electronic comic, means for setting customization options such as color, size, and material for the item selected by the user, means for displaying an image generated using a generative AI model based on the customization options, means for the user to check the generated image and confirm the order details, means for coordinating the confirmed order details with related vendors to manufacture and deliver the product, and means for analyzing the user's emotions in real time and suggesting customization options according to the emotions. This allows users to smoothly customize, check, and order products according to their emotions and preferences.

[1391] "User" refers to an individual or group that uses the system to customize and order items found in virtual spaces, electronic games, and electronic comics as real-world products.

[1392] "Virtual space" refers to a virtual environment that is different from the real world and is constructed on the Internet or a computer network.

[1393] An "electronic game" is a type of interactive electronic entertainment played using a computer or gaming console.

[1394] "Digital comics" are manga and graphic novels available in digital format.

[1395] "Items" refer to items or objects that users encounter in virtual spaces, electronic games, and electronic comics.

[1396] "Customization options" refers to the choices and settings such as color, size, material, etc. that a user sets for an item.

[1397] A "generative AI model" is a model that uses artificial intelligence technology to generate images and data based on customization options selected by the user.

[1398] The "emotion engine" is part of a system that analyzes users' emotions in real time and provides customization options and suggestions based on those emotions.

[1399] "Affiliated Businesses" refers to companies or corporations that receive orders confirmed by users and manufacture and deliver products.

[1400] "Order details" refers to details of the item customized by the user, and includes specific instructions and specifications for manufacturing and delivery.

[1401] "Back margin" refers to a commission or margin on sales that is paid by related parties to the system provider.

[1402] The present invention relates to a system that allows users to customize items found in virtual spaces, electronic games, and electronic comics to their liking and receive them as real-world products. This system incorporates an emotion engine that analyzes the user's emotions in real time, and provides customization options and suggestions based on the emotions. A specific implementation of the system is described below.

[1403] System configuration and hardware / software usage examples

[1404] This system mainly consists of three components: a server, a terminal, and a user. The server includes a database, a public web server, an API server, and a generative AI model. The terminal is a smartphone, tablet, or personal computer that users access. The emotion engine is a software module that performs real-time emotion analysis of user input data.

[1405] A user accesses the system

[1406] Terminal: The user accesses the system via a smartphone or PC browser, or a dedicated app. The user accesses the system's login page, enters their username and password, and presses the login button. Built-in security features ensure data safety.

[1407] Example: A user accesses "example.com / login", enters the username "user123" and password "password123", and clicks the login button.

[1408] Server: Receives login information and retrieves the corresponding user information from the database. It compares the retrieved information with the entered information, and if authentication is successful, generates the user's dashboard and returns it to the device.

[1409] Product Selection

[1410] Device: The user clicks the "Select a product" button on the dashboard to go to the category selection page, where they can select the desired product from the list of product categories offered.

[1411] Example: A user clicks the "Choose a Product" button on the dashboard and selects "Gaming Items" on the category page.

[1412] Server: Based on the selected category, retrieves a list of related products from the database and returns it to the terminal, where the user can view the product list.

[1413] Setting customization options

[1414] Terminal: The user inputs customization options such as color, size, material, etc. The emotion engine analyzes the user's emotions in real time and suggests customization options based on their preferences and emotions.

[1415] Example: A user sets the color of a "model sword" to "silver," the length to "50cm," and the material to "wood." The emotion engine analyzes the user's input and presents additional customization options.

[1416] Server: Receives the entered customization information and stores it in a temporary database.

[1417] Image generation and confirmation

[1418] Server: Based on the saved customization information, the server sends an image generation request to the generative AI model. The generative AI model generates an image that reflects the customization options and sends it back to the server.

[1419] Terminal: The generated image is displayed to the user. The emotion engine analyzes the user's reactions in real time and makes suggestions to readjust the image if necessary.

[1420] Example: A customized image sent from the generative AI model is displayed on the device, and the user can review it. The emotion engine will make adjustments as necessary.

[1421] Confirmation of order details

[1422] User: Check the generated image and if satisfied, click the "Confirm Order" button. The emotion engine may also display recommendations based on the user's emotions.

[1423] Server: Receives order confirmation information and stores it in the database as official order data. Generates order information for related vendors and communicates it to the vendors.

[1424] Cooperation with vendors

[1425] Server: Sends order information via the vendor's API or email system and receives order confirmation from the vendor.

[1426] Example: Use a vendor API to send order information and get order confirmation from the vendor.

[1427] User Notice and Receipt

[1428] Server: Receives notification of product manufacturing completion from the supplier and sends notification of product manufacturing completion to the user via email or app notification.

[1429] On the device: The user receives a notification and checks the product delivery status on the delivery status page.

[1430] User: After receiving the product, press the confirmation button on the system.

[1431] Example: After a user receives a product, they press the "Confirm Receipt" button in the system.

[1432] Closing process and back margin payment

[1433] Server: Periodically extracts order and sales information from the database, calculates back margins, and issues invoices to related vendors.

[1434] Example: The server calculates the back margin based on the order data and sends an invoice to the relevant vendor.

[1435] Prompt Sentence Examples

[1436] "Design a system that allows users to customize items using an emotion engine and experience the ordering process. Required functionality includes login, product selection, customization options, image generation using generative AI, ordering, vendor integration, notifications, receipt confirmation, and closing."

[1437] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1438] Step 1: Access the system

[1439] Terminal: The user accesses the system using a smartphone or PC browser, or a dedicated app. The user enters their username and password on the login page and presses the login button.

[1440] Input: Username "user123", Password "password123"

[1441] Output: User dashboard

[1442] Specific operation: The user accesses "example.com / login", enters the username and password, and clicks the login button.

[1443] Server: Receives login information and retrieves the corresponding user information from the database. The retrieved information is compared with the entered information, and if authentication is successful, a user dashboard is generated and returned to the terminal.

[1444] Input: Login information (username and password)

[1445] Output: Authentication results and user dashboard

[1446] Specific operation: The server retrieves user information from the database, performs authentication, and if successful, creates and returns a user dashboard.

[1447] Step 2: Select product category

[1448] On the device: The user clicks the "Select a product" button on the dashboard, which takes them to the category selection page. The user then selects the desired category from the list of product categories provided.

[1449] Input: Category selection request

[1450] Output: Product category page

[1451] Specific behavior: The user clicks the "Choose a Product" button on the dashboard and selects the "Game Items" category.

[1452] Server: Based on the selected category, retrieves a list of related products from the database and returns it to the terminal, which then displays the product list to the user.

[1453] Input: Category selection information

[1454] Output: Product list

[1455] Specific operation: The server retrieves data in the "game items" category from the database, sends a product list to the user's terminal, and displays it.

[1456] Step 3: Configure customization options

[1457] Terminal: The user inputs customization options such as color, size, material, etc. The emotion engine analyzes the user's emotions in real time and suggests customization options based on the user's preferences and emotions.

[1458] Input: Customization option information

[1459] Output: Optimized customization options

[1460] What it does: The user sets the color of the "model sword" to "silver," the length to "50cm," and the material to "wood." The emotion engine uses these inputs to suggest additional customization options.

[1461] Server: Receives the entered customization information and stores it in a temporary database.

[1462] Input: User-entered customization information

[1463] Output: Save results to a temporary database

[1464] Specific operation: The server receives the customization information of "silver, 50cm, wood" and stores it in a temporary database.

[1465] Step 4: Image generation and verification

[1466] Server: Based on the saved customization information, the server sends an image generation request to the generative AI model. The generative AI model generates an image that reflects the customization options and sends it back to the server.

[1467] Input: Customization information

[1468] Output: Generated customized image

[1469] Specific operation: The server requests the generative AI model to generate an image of a "silver, 50 cm, wooden sword" and receives the generated image data.

[1470] Terminal: The generated image is displayed to the user. The emotion engine analyzes the user's reactions in real time and makes suggestions to readjust the image if necessary.

[1471] Input: Customized image data

[1472] Output: Optimized customized image

[1473] Specific operation: The image sent from the generative AI model is displayed on the device, and the emotion engine suggests readjustments based on the user's emotional response.

[1474] Step 5: Confirm your order

[1475] User: Check the generated image and if satisfied, click the "Confirm Order" button. The emotion engine may also display recommendations based on the user's emotions.

[1476] Input: Order confirmation request

[1477] Output: Order confirmation notification

[1478] Specific operation: The user checks the image and clicks the "Confirm Order" button.

[1479] Server: Receives order confirmation information and stores it in the database as official order data. Generates order information for related vendors and communicates it to the vendors.

[1480] Input: Order confirmation information

[1481] Output: Official order data and order information to vendors

[1482] Specific operation: The server saves the order information in a database and prepares to send the order information to the supplier.

[1483] Step 6: Contact a vendor

[1484] Server: Sends order information via the vendor's API or email system and receives order confirmation from the vendor.

[1485] Input: Order data

[1486] Output: Vendor order confirmation

[1487] Specific operation: The server sends the order information to the supplier and receives an order confirmation.

[1488] Step 7: User Notification and Acceptance

[1489] Server: Receives notification from the supplier that product manufacturing is complete and sends notification of product manufacturing completion to the user via email or app notification.

[1490] Input: Product manufacturing completion notification

[1491] Output: Product manufacturing completion notification to user

[1492] Specific operation: The server receives the product manufacturing completion information and sends a notification to the user.

[1493] On the device: The user receives a notification and checks the product delivery status on the delivery status page.

[1494] Input: Receive completion notification

[1495] Output: Display delivery status

[1496] What happens: A user checks the delivery status of a product online.

[1497] User: After receiving the product, press the confirmation button on the system.

[1498] Input: Receipt Acknowledgment Request

[1499] Output: Notification of receipt

[1500] Specific operation: The user receives the product and clicks the "Confirm receipt" button.

[1501] Step 8: Closing and margin payment

[1502] Server: Periodically extracts order and sales information from the database, calculates back margins, and issues invoices to related vendors.

[1503] Input: Order and sales data

[1504] Output: Back margin calculation results and invoice

[1505] Specific operation: The server calculates the back margin based on the order data for the specified period and sends an invoice to the relevant vendor.

[1506] (Application example 2)

[1507] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1508] In current systems that allow users to receive items seen in virtual spaces or games as physical goods in the real world, the customization experience is complicated, making it difficult to achieve highly satisfying customization that reflects the user's emotions.Another issue is that the wide range of customization options makes it difficult for users to make the optimal choice.

[1509] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1510] In this invention, the server includes: a means for allowing a user to select an item seen in a virtual space, game, or comic; a means for setting customization options such as color, size, and material for the item selected by the user; an emotion analysis means for analyzing the user's emotions in real time and suggesting customization options; a means for displaying an image generated based on the customization options; a means for the user to check the generated image and confirm the order details; and a means for coordinating the confirmed order details with related vendors to have the product manufactured and delivered. This enables a highly satisfying customization and ordering process that reflects the user's emotions.

[1511] A "virtual space" is a virtual environment generated by a computer.

[1512] A "game" is an interactive activity under prescribed rules for recreational or competitive purposes.

[1513] "Comics" is a form of publication that tells a story through pictures and text.

[1514] An "item" is a thing or element that exists in relation to a particular use or purpose.

[1515] "Customization options" are options that allow users to set the color, size, material, etc. of an item according to their preferences.

[1516] "Emotion analysis means" is a technology that analyzes emotions in real time from a user's facial expressions and behavior.

[1517] A "generated image" is a visual representation generated based on the customization options set by the user.

[1518] "Order details" refers to product specifications and order information confirmed by the user.

[1519] "Affiliated businesses" are companies or organizations that manufacture and deliver products based on user orders.

[1520] A "generative AI model" is an algorithm that uses artificial intelligence to generate images or text from data.

[1521] A "system" is a set of processes or devices consisting of multiple elements configured to achieve a specific purpose.

[1522] This invention provides a customization and ordering process that responds to the user's emotions by combining an emotion analysis engine with a system that allows users to customize items they see in virtual spaces, games, and comics and receive them as real products. The following describes an embodiment of the invention.

[1523] Hardware and software used

[1524] 1. Smartphone: Users use iOS or Android devices, and a dedicated application is installed on the smartphone.

[1525] 2. Emotion Analysis Engine: Affectiva SDK is used. This SDK analyzes the user's facial expression data and grasps their emotional state in real time.

[1526] 3. Generative AI model: OpenAI GPT and Gemini AI are used. OpenAI GPT performs natural language analysis and generation, and Gemini AI generates images based on customization options.

[1527] 4. Server: The server uses AWS (Amazon Web Services), which handles back-end processing, data storage, and vendor integration.

[1528] 5. Database: MySQL and DynamoDB are used to store customization information, order information, user information, etc.

[1529] 6. Notification system: Use Firebase Cloud Messaging (FCM) to send notifications to users.

[1530] 7. API communication: Data communication between systems is carried out using RESTful APIs.

[1531] Specific examples of the invention

[1532] A user accesses the system

[1533] Users access the system from a smartphone application and log in by entering their username and password. The server receives the login information, retrieves user information from the database, and performs authentication. If authentication is successful, it returns the user dashboard to the device.

[1534] The user selects the product they want

[1535] The user clicks the "Select a Product" button on the smartphone dashboard, which takes them to a category selection page. The user selects "Game Items" and then "Model Sword" from the list. The server retrieves a list of related products from the database based on the selected category and returns it to the device. The user then checks the product list on their smartphone.

[1536] Set customization options

[1537] Users input customization options such as color, size, and material. At this time, the sentiment analysis engine analyzes the user's emotions in real time and suggests customization options based on the user's preferences and emotions. For example, if a user sets the color of a "model sword" as silver, the length as 50 cm, and the material as wood, the sentiment analysis engine will suggest better options.

[1538] Image generation and confirmation using Gemini

[1539] The server sends an image generation request to Gemini AI, a generative AI model, based on the saved customization information. The image data of the customized product returned by Gemini AI is then received and returned to the device. The user can then view the generated image on their smartphone, and the sentiment analysis engine analyzes the user's reactions in real time, readjusting the image as necessary.

[1540] Confirmation of order details

[1541] The user checks the generated image and clicks the "Confirm Order" button to confirm the order. At this time, the sentiment analysis engine displays recommendations based on the user's emotions. The server receives the order confirmation information and stores it in the database as official order data. It also generates order information for related vendors.

[1542] Cooperation with vendors

[1543] The server sends the order information via the supplier's API or email system, receives an order confirmation from the supplier, and the supplier starts the manufacturing process based on the order information.

[1544] User Notice and Receipt

[1545] When the server receives a notification from the supplier that the product has been completed, it will notify the user by email or app notification. The user will receive the notification and check the product's delivery status on the delivery status page. When the user receives the product, they can press the receipt confirmation button on the system to complete the receipt.

[1546] Closing process and back margin payment

[1547] The server periodically extracts order and sales information from the database, calculates the back margin, and issues invoices to related vendors, allowing the system provider to earn revenue.

[1548] Prompt Sentence Examples

[1549] Below is an example prompt that can be input to a generative AI model to generate suggested customization options based on sentiment analysis.

[1550] text

[1551] The user selected the "Sword Model." Based on the user's facial expression data, their current emotional state is "Happy." Suggest the best customization options for this sword model based on the emotion of "Happy." Recommend color, material, and design changes as needed.

[1552] This prompt provides emotional data to the generative AI model, which then suggests appropriate customization options. This approach allows users to customize products in a way that best suits their emotions, resulting in a satisfying shopping experience.

[1553] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1554] Step 1:

[1555] A user accesses the system and enters their login information.

[1556] Input: Username and Password

[1557] Specific operation: The user accesses the login screen of the smartphone app, enters their username and password, and presses the login button.

[1558] Data processing: The server receives the entered login information and retrieves user information from the database (MySQL).

[1559] Output: If authentication is successful, returns the user dashboard to the device.

[1560] Step 2:

[1561] The user selects the product they want.

[1562] Input: "Select a product" button and category selection (e.g., game items, sword model)

[1563] Specific Action: The user clicks the "Select a Product" button on the dashboard, goes to the category selection page, and then selects a specific item (e.g., "Model Sword") from the selected category.

[1564] Data processing: The server retrieves a list of related products from the database (DynamoDB) based on the selected category.

[1565] Output: The terminal displays the product list to the user.

[1566] Step 3:

[1567] Enter the customization options.

[1568] Input: Customization information such as item color, size, material, etc.

[1569] How it works: Users input customization options for the selected item, such as color, size, material, etc. At the same time, the emotion analysis engine (Affectiva SDK) analyzes the user's facial expression data in real time.

[1570] Data processing: The server sends the sentiment analysis data to OpenAI GPT, which then suggests customization options based on the user's emotions.

[1571] Output: Prints the suggested customization options to the terminal.

[1572] Step 4:

[1573] Check the generated image.

[1574] Input: Customization information and user emotion data

[1575] Specific operation: The user confirms the settings based on the customization options. After the settings are confirmed, the server sends an image generation request to Gemini AI based on the settings.

[1576] Data processing: Gemini AI creates a generated image based on the customization information and sends it back to the server.

[1577] Output: The server returns the generated image to the device, which displays it to the user. The sentiment analysis engine then analyzes the user's reaction again and adjusts the image if necessary.

[1578] Step 5:

[1579] Confirm the order details.

[1580] Input: Confirmed customizations and generated images

[1581] Specific operation: The user checks the generated image and clicks the "Confirm Order" button. At this time, the sentiment analysis engine displays recommendations based on the user's emotions.

[1582] Data processing: The server receives the order confirmation information, stores it in the database as official order data, and generates order information for related vendors.

[1583] Output: The order information is sent to the relevant supplier and the manufacturing process is initiated.

[1584] Step 6:

[1585] Product manufacturing and user notification.

[1586] Input: Production completion notice from supplier

[1587] Specific operation: The server receives notification from the relevant supplier that the product has been completed. Based on this information, it uses Firebase Cloud Messaging (FCM) to notify the user that the product has been completed.

[1588] Data processing: Generates notification information and sends it to the user's device.

[1589] Output: The user receives a notification on their smartphone that production is complete and checks the delivery status on the delivery status page.

[1590] Step 7:

[1591] Receipt and confirmation in the system.

[1592] Input: When the user receives the product, they press the "Confirm Receipt" button.

[1593] Specific operation: After receiving the product, the user presses the "Confirm receipt" button on the smartphone app, which causes the system to confirm receipt.

[1594] Data processing: The server records the received data and stores it in a database.

[1595] Output: The receipt confirmation information is saved in the database and the product receipt procedure is completed.

[1596] Step 8:

[1597] Regular data processing and back margin calculation.

[1598] Input: Order and sales information in the database

[1599] Specific operation: The server periodically extracts order information and sales information from the database, calculates the back margin based on this information, and issues invoices to related vendors.

[1600] Data processing: Analyze and calculate sales data and order information.

[1601] Output: Invoices are sent to the relevant vendors and back margin payments are processed.

[1602] As described above, by clearly indicating the specific operations, inputs, and outputs at each step, the user can smoothly complete the customization and ordering process and achieve a high level of satisfaction.

[1603] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1604] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1605] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[1606] [Fourth embodiment]

[1607] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[1608] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1609] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1610] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[1611] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1612] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1613] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1614] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[1615] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1616] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1617] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1618] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1619] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1620] This invention relates to a system that allows users to customize and order items they see in virtual spaces, games, and comics, and then receive them as real-world products. This system allows users to freely customize specific items to their liking and easily order and purchase them. The program processing of this system is explained below with specific examples.

[1621] A user accesses the system

[1622] Terminal: The user accesses the system via a browser or a dedicated app.

[1623] Example: A user uses a smartphone to access the system's login page, enters their username and password, and presses the login button.

[1624] Server: Receives login information and performs authentication.

[1625] The server retrieves user information from the database, verifies the entered username and password, and if authentication is successful, returns the dashboard screen to the terminal.

[1626] The user selects the product they want

[1627] Terminal: User selects product category and chooses specific items.

[1628] Example: A user selects "game items" and then selects "sword model."

[1629] Server: Serves a list of products based on the selected category.

[1630] The server retrieves products related to the "game item" from the database and returns the list to the terminal, which then displays the product list to the user.

[1631] Set customization options

[1632] Terminal: The user sets the product's color, size, material, etc.

[1633] Example: A user sets the color of a "sword model" to silver, the length to 50 cm, and the material to wood.

[1634] Server: Receives and temporarily saves the configured customization options.

[1635] The server temporarily stores the received customization information in a database.

[1636] Image generation and confirmation using Gemini

[1637] Server: Sends customization information to Gemini AI and generates images.

[1638] The server calls the Gemini API based on the stored customization information to generate an image of the customized item.

[1639] Terminal: displays the generated image to the user.

[1640] Example: Image data sent from Gemini AI is displayed on the device, and the user checks the image.

[1641] Confirmation of order details

[1642] User: Check the generated image and confirm the order details.

[1643] If the user is satisfied with the image, he / she presses the "Confirm Order" button to confirm the order.

[1644] Server: Stores the order details in a database and generates information to be shared with related vendors.

[1645] The server officially stores the order details in a database and transmits the order information to related vendors.

[1646] Cooperation with vendors

[1647] Server: Sends order information to related vendors.

[1648] The server sends the order information via the vendor's API and waits for order confirmation from the vendor.

[1649] Supplier: Starts production based on the order information.

[1650] Example: A vendor receives an order and manufactures a model sword, preparing it for delivery.

[1651] User Notice and Receipt

[1652] Server: Notify the user that the product is complete.

[1653] The server receives a notification of production completion from the supplier and notifies the user via email or app notification.

[1654] Terminal: User receives notification and checks delivery status.

[1655] Users receive notifications and view progress on the delivery status page.

[1656] User: Receives the product and confirms receipt.

[1657] Example: After receiving the product, the user presses the receipt confirmation button on the system.

[1658] Closing process and back margin payment

[1659] Server: Calculates back margins, closes the transaction, and bills the sales company.

[1660] The server periodically extracts order and sales information from the database, calculates the back margin based on the contract with the related vendor, and issues invoices, which allows the system provider to earn revenue.

[1661] The processing flow will be explained below.

[1662] Step 1:

[1663] Terminal: The user accesses the system and moves to the login screen. They enter their username and password and press the login button.

[1664] Server: Receives login information, retrieves user information from the database, and performs authentication. After successful authentication, returns the user dashboard to the terminal.

[1665] Step 2:

[1666] Device: User clicks the "Choose a Product" button from the dashboard. The category selection page appears.

[1667] Device: The user selects a product category, for example, "Game Items."

[1668] Server: Based on the selected category, retrieves a list of related products from the database and returns it to the terminal, which then displays the product list to the user.

[1669] Step 3:

[1670] Terminal: The user selects a specific product from the list, such as a "model sword," and clicks the "Customize" button.

[1671] Device: A customization screen appears, allowing the user to enter customization options such as color, size, and material.

[1672] Server: Receives the entered customization information and stores it in a temporary database.

[1673] Step 4:

[1674] Server: Sends an image generation request to Gemini AI based on the saved customization information.

[1675] Server: Receives the image data of the customized product returned from Gemini AI and returns it to the terminal.

[1676] Terminal: Displays customized product images to the user.

[1677] Step 5:

[1678] Terminal: The user checks the generated image and clicks the "Confirm Order" button to confirm the order.

[1679] Server: Receives order confirmation information and stores it in the database as official order data. Generates order information for related vendors.

[1680] Step 6:

[1681] Server: Sends order information via the vendor's API or email system. Receives order confirmation from the vendor.

[1682] Vendor: Starts the manufacturing process of the product based on the received order information.

[1683] Step 7:

[1684] Server: Receives notification of product manufacturing completion from the supplier. Sends notification of product manufacturing completion to the user via email or app notification.

[1685] On the device: The user receives a notification and checks the delivery status of the product on the delivery status page.

[1686] Step 8:

[1687] User: Receives the product and presses the receipt confirmation button on the system.

[1688] Server: Saves the receipt confirmation information in a database.

[1689] Step 9:

[1690] Server: Periodically extracts order and sales information from the database, calculates back margins, and issues invoices to related vendors.

[1691] Example 1

[1692] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1693] Conventional online shopping systems make it difficult for users to customize and order digital items as real physical products. Furthermore, it is difficult for users to visually confirm the customized items, and there are many cases where the product does not meet expectations after ordering. Furthermore, there are problems with efficiently calculating back margins and billing procedures linked to the product manufacturing process.

[1694] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1695] In this invention, the server includes a means for allowing a user to select a digital item to be seen in a virtual space or entertainment content, a means for setting customization options such as color, size, and material for the selected digital item, and a means for displaying an image generated using a generative AI model based on the customization options. This allows a user to easily customize a digital item while visually checking it and then order it as a real physical product. Furthermore, by linking with the manufacturing process, back margin calculations and billing procedures can be efficiently performed.

[1696] "User" means an individual or entity that utilizes the System to select, customize, and order Digital Items as physical products.

[1697] "Digital items" refer to various products and objects that are visually recognized in virtual spaces and entertainment content.

[1698] "Customization options" are options that allow a user to change the color, size, material, etc. of a digital item selected by the user according to the user's preferences.

[1699] A "generative AI model" is an artificial intelligence algorithm for generating images of a digital item based on customization options.

[1700] "Image generation" is the process of using a generative AI model to create a visual display based on customization options.

[1701] A "manufacturing organization" is a company or factory that receives orders from users, manufactures physical products, and delivers them.

[1702] A "physical product" is a physical product that is created by physically creating a digital item.

[1703] "Back margin" refers to the commission or profit sharing that a system provider receives from a manufacturing organization based on a contract with the manufacturing organization for orders or sales.

[1704] "Notification means" refers to the method by which the system keeps users informed about product completion and delivery status.

[1705] A "prompt sentence" is text data input into a generative AI model, and contains customization information for image generation.

[1706] This invention relates to a system that allows users to select digital items that appear in virtual spaces or entertainment content, customize them, and order them as physical products. This system realizes a smooth process from customization to ordering and product delivery through mutual cooperation between the user, terminal, and server.

[1707] First, users access the system via a smartphone or computer browser or a dedicated app. They access the system's login page and log in by entering their username and password. After that, the dashboard screen is displayed, allowing the user to freely operate the system.

[1708] Next, the user selects a product category from the displayed dashboard and then selects a specific digital item from that category. For example, the user can select "Game Items" and then select a "Sword Model." For the selected digital item, the user can set customization options such as color, size, and material.

[1709] Once customization is complete, the server uses a generative AI model such as Gemini to generate an image based on the customization options. This generated image is then displayed to the user via their device. The user can review the displayed image and customize it again if necessary.

[1710] If the user is satisfied with the generated image, they press the "Confirm Order" button to confirm the order. The server officially saves this order in the database and shares the information with the manufacturing organization. At this point, the manufacturing organization begins manufacturing the physical product based on the received customization information.

[1711] Once the product is completed, the manufacturing organization sends the completion information to the server, and the server notifies the user. The user receives the notification and can check the delivery status. Once the product is delivered to the user, the user confirms receipt in the system and the process is complete. Finally, the server calculates the back margin based on the order information and bills the relevant organization.

[1712] For example, suppose a user selects a "model sword," sets the color to silver, the length to 50cm, and the material to wood, and requests that an image be generated. An example prompt for this would be:

[1713] Example prompt sentence:

[1714] "The user has selected a model sword and specified customization options: color is silver, length is 50cm, and material is wood. Use this information to generate an image of the customized model sword."

[1715] As a result, the system of the present invention provides an efficient and intuitive platform that allows users to easily customize digital items visually and order them as real physical products, while also realizing monetization for the system provider by automating the manufacturing process and back-margin calculation.

[1716] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1717] Step 1: User Login

[1718] Input: Username and Password

[1719] Specific operation: The user accesses the system's login page using a browser on their device (smartphone or computer) or a dedicated app, enters their username and password, and presses the login button.

[1720] Data processing and calculation: The server receives the entered username and password, retrieves the user information from the database, and performs verification.

[1721] Output: If the match is successful, the dashboard screen is returned to the terminal, allowing the user to operate the system.

[1722] Step 2: Select your product

[1723] Input: User selection of product category and specific item

[1724] Specific operation: The user selects a product category (e.g., "game items") from the dashboard and then selects a specific item (e.g., "sword model") from that category.

[1725] Data processing and calculation: The server receives the user's selection information and retrieves the corresponding product list from the database.

[1726] Output: The server returns the product list to the terminal, which displays the list to the user.

[1727] Step 3: Configure customization options

[1728] Input: Customization options such as color, size, material, etc. set by the user.

[1729] Specific operation: The user sets the color of the selected "sword model" to silver, the length to 50cm, and the material to wood.

[1730] Data processing and calculation: The server receives the customization options and temporarily stores them in the database.

[1731] Output: The customization information is saved in the database.

[1732] Step 4: Generate customized images

[1733] Input: Customization option information

[1734] Specific operation: The server calls the API of a generative AI model (e.g., Gemini) based on the saved customization information to generate an image of the customized item. It sends the following prompt: "The user selected a sword model and specified customization options. The color is silver, the length is 50 cm, and the material is wood. Please generate an image of the customized sword model based on this information."

[1735] Data processing and calculation: The server generates an image using the generative AI model and receives the image data.

[1736] Output: The generated image data is sent from the server to the terminal and displayed.

[1737] Step 5: Confirm your order

[1738] Input: User's order confirmation instructions

[1739] Specific operation: The user checks the generated image and presses the "Confirm Order" button if satisfied.

[1740] Data processing and calculation: The server officially stores the order details in the database and generates information linked to the manufacturing organization.

[1741] Output: The purchase order is sent to the manufacturing organization and production of the product begins.

[1742] Step 6: Product Completion Notification and Delivery Confirmation

[1743] Input: Product completion notification from manufacturing organization

[1744] Specific operation: The server receives a product completion notification from the manufacturing organization and notifies the user.

[1745] Data processing and calculation: The server sends the product completion information to the user via email or app notification.

[1746] Output: User receives notification and checks delivery status.

[1747] Step 7: Acknowledgement

[1748] Input: User's receipt confirmation instructions

[1749] Specific operation: After receiving the product, the user presses the receipt confirmation button on the system.

[1750] Data processing and calculation: The server saves the receipt confirmation data in the database.

[1751] Output: The receipt confirmation information is saved in the system.

[1752] Step 8: Calculate back margin

[1753] Input: Order and sales information

[1754] Specific operation: The server periodically extracts order and sales information from the database.

[1755] Data processing and calculation: The server calculates the back margin based on the order and sales information and issues an invoice.

[1756] Output: The calculation results and invoices are sent to the relevant organizations, and the system provider receives revenue.

[1757] (Application example 1)

[1758] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1759] In conventional systems where users receive items seen in virtual spaces, games, or comics as real-world products, it was sometimes difficult to manufacture and deliver products that accurately reflected the user's customizations. Furthermore, there was a lack of means for users to check the items they customized, and insufficient collaboration with vendors, making it difficult to improve the user experience. Furthermore, improving the customization experience in virtual stores using smart devices was also a challenge.

[1760] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1761] In this invention, the server includes a means for allowing a user to select an item seen in a virtual reality space, electronic game, or comic; a means for setting customization options, such as color, size, and material, for the selected item; and a means for displaying an image generated based on the customization options. This allows the user to browse a virtual store and customize the item via a smart device. The server also includes a means for confirming the generated image and confirming order details, and a means for linking the confirmed order details to a manufacturer and having the product manufactured and delivered. This ensures that products reflecting the user's customization are manufactured and delivered. Furthermore, the server includes a means for generating an image based on the customization options using a generative AI model and a means for using prompt sentences, enabling the server to quickly and accurately generate an image that accurately reflects the user's customization, improving the user experience.

[1762] "User" refers to an individual who selects and customizes an item seen in a virtual reality space, electronic game, or comic, and then checks the generated image and places an order.

[1763] A "virtual reality space" is a virtual environment created using computer or digital technology that users can interact with.

[1764] An "electronic game" is an interactive entertainment system that is implemented using digital devices.

[1765] "Manga" is content that combines narrative and illustrations using visual storytelling.

[1766] "Items" is a general term for items that users encounter in virtual reality spaces, electronic games, and comics.

[1767] "Customization options" refer to setting items that a user can specify for the item they have selected, including color, size, material, and the like.

[1768] "Generated Image" means a visual representation generated by an AI generative model based on customization options set by a user.

[1769] "Order details" are detailed information that allows the user to check the generated image and confirm the purchase.

[1770] A "manufacturer" is a company or organization that receives a user's order, manufactures the actual goods, and delivers them.

[1771] A "smart device" is an electronic device that can be carried by a user and that can connect to the Internet and run applications, and includes smartphones, smart glasses, etc.

[1772] A "virtual store" is a virtual shopping area built in a virtual reality space where users can browse, customize, and order items.

[1773] A "generative AI model" is an artificial intelligence algorithm used to generate an image of an item based on a user's customization options.

[1774] A "prompt" is textual input that instructs a generative AI model to generate a specific image.

[1775] This invention relates to a system that allows users to select an item they see in a virtual reality space, an electronic game, or a comic, customize it, and receive it as a real product. This system allows users to customize the item to their liking and easily order and purchase it. The program processing of this system is explained in detail below.

[1776] Hardware and Software Configuration

[1777] The system uses the following hardware and software:

[1778] Smart devices: Used by users to access and customize virtual spaces. Specifically, these include smartphones and smart glasses.

[1779] Server: Receives requests from users and performs authentication, data processing, image generation, and coordination of order details.

[1780] Flask: Used as a server-side web application framework.

[1781] MongoDB: Used as a database to store user information, product information, customization information, and order information.

[1782] Generative AI Model: The AI ​​model used to generate images of customized items. Specifically, it utilizes the Gemini AI API.

[1783] System processing flow

[1784] 1. User login: The user accesses the system using a smart device and enters their username and password on the login page for authentication. The server retrieves the user information from MongoDB and performs authentication.

[1785] 2. Product category selection: After successful authentication, the user selects a product category, for example, "Sword model" from the "Game items" category.

[1786] 3. Setting customization options: The user sets customization options for the selected item, such as color, size, material, etc. For example, the color of the "model sword" can be silver, the length can be 50 cm, and the material can be wood.

[1787] 4. Image Generation: The server calls the generative AI model (Gemini AI) based on the customization information and generates an image of the customized item. The generated image is displayed to the user.

[1788] 5. Confirmation of order: The user checks the generated image and confirms the order details. The server saves the order details in the database and generates information to be shared with related vendors.

[1789] 6. Linking to vendor: The server sends the order information via the vendor's API and waits for the order confirmation from the vendor. The vendor starts production based on the received order information.

[1790] 7. Notification and Delivery: The server receives the completed product and notifies the user via email or app notification. The user can check the progress on the card authentication page.

[1791] Specific prompt examples

[1792] Here are some example prompts to input to a generative AI model:

[1793] Generate images for the following customized items:

[1794] Item: Model sword

[1795] Color: Blue

[1796] Length: 60cm

[1797] Material: Iron

[1798] In this way, this system allows users to customize items they see in a virtual space and receive them as real products. By combining smart devices with advanced AI technology, it improves the user experience and realizes a smooth ordering, manufacturing, and delivery process.

[1799] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1800] Step 1:

[1801] A user accesses the system using a smart device and enters login information (user name and password).

[1802] Input: Username, Password

[1803] Data processing: The smart device sends the user name and password to the server, which searches for the corresponding user information in MongoDB and performs authentication.

[1804] Output: Authentication result (success / failure)

[1805] Specific operation: The server checks whether the username and password match, and if successful, displays the dashboard screen to the user.

[1806] Step 2:

[1807] The user selects a product category from the dashboard screen.

[1808] Input: Selected Product Category

[1809] Data processing: The selected category information is sent from the smart device to the server, which then retrieves the corresponding product list from MongoDB.

[1810] Output: Product list

[1811] Specific operation: The server displays the retrieved product list on the user's smart device, and the user selects a specific item from the list.

[1812] Step 3:

[1813] The user sets customization options (color, size, material, etc.) for the selected item.

[1814] Input: Customization options (color, size, material)

[1815] Data processing: Customization information is sent from the smart device to the server, which temporarily stores it in MongoDB.

[1816] Output: Save result (success / failure)

[1817] Specific operation: The server checks the received customization information, and if it is in the correct format, it temporarily saves it and proceeds to the next image generation process.

[1818] Step 4:

[1819] The server calls the generative AI model based on the saved customization information and generates an image of the customized item.

[1820] Input: Customization information

[1821] Data processing: The server sends the customization information as a prompt to the generative AI model, which then generates an image based on the customization information.

[1822] Output: The generated image

[1823] Specific operation: The server generates a prompt sentence in the following format and sends it to the generative AI model.

[1824] Generate images for the following customized items:

[1825] Item: Model sword

[1826] Color: Blue

[1827] Length: 60cm

[1828] Material: Iron

[1829] The generated image data is received and displayed to the user.

[1830] Step 5:

[1831] The user checks the generated image and confirms the order details.

[1832] Input: User confirmation (order confirmation)

[1833] Data processing: The user's confirmation is sent to the server. The server receives this, officially stores the order details in MongoDB, and prepares to send them to the relevant vendors.

[1834] Output: Order confirmation notice, order information

[1835] Specific operation: When the user presses the "Confirm Order" button, the server officially saves the order details and communicates with related vendors.

[1836] Step 6:

[1837] The server uses the vendor's API to send the order information.

[1838] Input: Order Information

[1839] Data processing: The server sends the order information to the vendor's API and receives an order confirmation.

[1840] Output: Order confirmation

[1841] Specific operation: The server sends the order information to the supplier's system and stores the order confirmation from the supplier in MongoDB.

[1842] Step 7:

[1843] The server receives the completion of the product and notifies the user.

[1844] Input: Notification of completion of production from supplier

[1845] Data processing: The server stores the received manufacturing completion notification in MongoDB and notifies the user via email or app notification.

[1846] Output: User notification

[1847] Specific operation: Based on the notification from the supplier, the server sends a notification to the user that the product has been completed, and the user can check the delivery status.

[1848] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1849] This invention combines an emotion engine with a system that allows users to customize and order items they see in virtual spaces, games, and comics, and then receive them as real-world products, thereby providing a customization and ordering process that responds to the user's emotions. Below, we will explain the program processing of this system using concrete examples.

[1850] A user accesses the system

[1851] Terminal: The user accesses the system via a browser or a dedicated app.

[1852] Example: A user uses a smartphone to access the system's login page, enters their username and password, and presses the login button.

[1853] Server: Receives login information, retrieves user information from the database, and performs authentication. After successful authentication, returns the user dashboard to the terminal.

[1854] The user selects the product they want

[1855] Device: The user clicks the "Select a product" button on the dashboard and is taken to the category selection page.

[1856] Example: A user selects "game items" and then selects "sword model."

[1857] Server: Based on the selected category, retrieves a list of related products from the database and returns it to the terminal, which then displays the product list to the user.

[1858] Set customization options

[1859] Device: The user inputs customization options such as color, size, material, etc. In addition, the emotion engine analyzes the user's emotions in real time and suggests customization options based on the user's preferences and emotions.

[1860] Example: A user sets the color of a "model sword" as silver, the length as 50 cm, and the material as wood, and consults the options suggested by the emotion engine.

[1861] Server: Receives the entered customization information and stores it in a temporary database.

[1862] Image generation and confirmation using Gemini

[1863] Server: Sends an image generation request to Gemini AI based on the saved customization information.

[1864] The server receives the image data of the customized product returned by Gemini AI and returns it to the terminal.

[1865] Terminal: The generated image is displayed to the user. The emotion engine analyzes the user's reaction in real time and readjusts the image as needed.

[1866] Example: Image data sent from Gemini AI is displayed on the device, and after the user checks the image, the emotion engine makes readjustments.

[1867] Confirmation of order details

[1868] User: Check the generated image and click the "Confirm Order" button to confirm the order.

[1869] The emotion engine displays recommendations based on the user's emotions.

[1870] Server: Receives order confirmation information and stores it in the database as official order data. Generates order information for related vendors.

[1871] Cooperation with vendors

[1872] Server: Sends order information via the vendor's API or email system. Receives order confirmation from the vendor.

[1873] The supplier starts the manufacturing process of the product based on the received order information.

[1874] User Notice and Receipt

[1875] Server: Receives notification of product manufacturing completion from the supplier. Sends notification of product manufacturing completion to the user via email or app notification.

[1876] On the device: The user receives a notification and checks the delivery status of the product on the delivery status page.

[1877] User: Receives the product and presses the receipt confirmation button on the system.

[1878] Example: After receiving the product, the user presses the receipt confirmation button in the system.

[1879] Closing process and back margin payment

[1880] Server: Periodically extracts order and sales information from the database, calculates back margins, and issues invoices to related vendors.

[1881] The server periodically calculates the back margin based on the order data and sends invoices to the relevant vendors. This process allows the system provider to earn revenue.

[1882] This allows users to easily select and confirm customized products that suit their preferences, and the emotion engine can provide an even more satisfying purchasing experience.

[1883] The processing flow will be explained below.

[1884] Step 1:

[1885] Terminal: The user accesses the system via a browser or a dedicated app, goes to the login screen, enters their username and password, and presses the login button.

[1886] Server: Receives login information, retrieves user information from the database, and performs authentication. After successful authentication, returns the user dashboard to the terminal.

[1887] Step 2:

[1888] Device: The user clicks the "Select a product" button on the dashboard and is taken to the category selection page.

[1889] Terminal: The user selects a product category, for example, "game items."

[1890] Server: Based on the selected category, retrieves a list of related products from the database and returns it to the terminal, which then displays the product list to the user.

[1891] Step 3:

[1892] Terminal: The user selects a specific product from the list, such as a "model sword," and clicks the "Customize" button.

[1893] Device: A customization screen is displayed, allowing the user to input customization options such as color, size, material, etc. Furthermore, an emotion engine analyzes the user's emotions in real time and suggests customization options based on the user's preferences and emotions.

[1894] Server: Receives the entered customization information and stores it in a temporary database.

[1895] Step 4:

[1896] Server: Sends an image generation request to Gemini AI based on the saved customization information.

[1897] Server: Receives the image data of the customized product returned from Gemini AI and returns it to the terminal.

[1898] Terminal: The generated image is displayed to the user. The emotion engine analyzes the user's reaction in real time and readjusts the image as needed.

[1899] Example: Image data sent from Gemini AI is displayed on the device, and after the user checks the image, the emotion engine makes readjustments.

[1900] Step 5:

[1901] Terminal: The user checks the generated image and clicks the "Confirm Order" button to confirm the order. The emotion engine displays recommended content based on the user's emotions.

[1902] Server: Receives order confirmation information and stores it in the database as official order data. Generates order information for related vendors.

[1903] Step 6:

[1904] Server: Sends order information via the vendor's API or email system. Receives order confirmation from the vendor.

[1905] Vendor: Starts the manufacturing process of the product based on the received order information.

[1906] Step 7:

[1907] Server: Receives notification of product manufacturing completion from the supplier. Sends notification of product manufacturing completion to the user via email or app notification.

[1908] On the device: The user receives a notification and checks the delivery status of the product on the delivery status page.

[1909] Step 8:

[1910] User: Receives the product and presses the receipt confirmation button on the system.

[1911] Server: Saves the receipt confirmation information in a database.

[1912] Step 9:

[1913] Server: Periodically extracts order and sales information from the database, calculates back margins, and issues invoices to related vendors.

[1914] Example 2

[1915] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1916] Currently, when users customize and order real-world products based on items they see in virtual spaces, electronic games, or electronic comics, it is difficult to achieve high satisfaction because there is a lack of suggestions that reflect the user's emotions and preferences. Another problem is that the purchase process is hindered by the cumbersome process of readjusting the customized images after viewing them.

[1917] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1918] In this invention, the server includes means for allowing a user to select an item seen in a virtual space, electronic game, or electronic comic, means for setting customization options such as color, size, and material for the item selected by the user, means for displaying an image generated using a generative AI model based on the customization options, means for the user to check the generated image and confirm the order details, means for coordinating the confirmed order details with related vendors to manufacture and deliver the product, and means for analyzing the user's emotions in real time and suggesting customization options according to the emotions. This allows users to smoothly customize, check, and order products according to their emotions and preferences.

[1919] "User" refers to an individual or group that uses the system to customize and order items found in virtual spaces, electronic games, and electronic comics as real-world products.

[1920] "Virtual space" refers to a virtual environment that is different from the real world and is constructed on the Internet or a computer network.

[1921] An "electronic game" is a type of interactive electronic entertainment played using a computer or gaming console.

[1922] "Digital comics" are manga and graphic novels available in digital format.

[1923] "Items" refer to items or objects that users encounter in virtual spaces, electronic games, and electronic comics.

[1924] "Customization options" refers to the choices and settings such as color, size, material, etc. that a user sets for an item.

[1925] A "generative AI model" is a model that uses artificial intelligence technology to generate images and data based on customization options selected by the user.

[1926] The "emotion engine" is part of a system that analyzes users' emotions in real time and provides customization options and suggestions based on those emotions.

[1927] "Affiliated Businesses" refers to companies or corporations that receive orders confirmed by users and manufacture and deliver products.

[1928] "Order details" refers to details of the item customized by the user, and includes specific instructions and specifications for manufacturing and delivery.

[1929] "Back margin" refers to a commission or margin on sales that is paid by related parties to the system provider.

[1930] The present invention relates to a system that allows users to customize items found in virtual spaces, electronic games, and electronic comics to their liking and receive them as real-world products. This system incorporates an emotion engine that analyzes the user's emotions in real time, and provides customization options and suggestions based on the emotions. A specific implementation of the system is described below.

[1931] System configuration and hardware / software usage examples

[1932] This system mainly consists of three components: a server, a terminal, and a user. The server includes a database, a public web server, an API server, and a generative AI model. The terminal is a smartphone, tablet, or personal computer that users access. The emotion engine is a software module that performs real-time emotion analysis of user input data.

[1933] A user accesses the system

[1934] Terminal: The user accesses the system via a smartphone or PC browser, or a dedicated app. The user accesses the system's login page, enters their username and password, and presses the login button. Built-in security features ensure data safety.

[1935] Example: A user accesses "example.com / login", enters the username "user123" and password "password123", and clicks the login button.

[1936] Server: Receives login information and retrieves the corresponding user information from the database. It compares the retrieved information with the entered information, and if authentication is successful, generates the user's dashboard and returns it to the device.

[1937] Product Selection

[1938] Device: The user clicks the "Select a product" button on the dashboard to go to the category selection page, where they can select the desired product from the list of product categories offered.

[1939] Example: A user clicks the "Choose a Product" button on the dashboard and selects "Gaming Items" on the category page.

[1940] Server: Based on the selected category, retrieves a list of related products from the database and returns it to the terminal, where the user can view the product list.

[1941] Setting customization options

[1942] Terminal: The user inputs customization options such as color, size, material, etc. The emotion engine analyzes the user's emotions in real time and suggests customization options based on their preferences and emotions.

[1943] Example: A user sets the color of a "model sword" to "silver," the length to "50cm," and the material to "wood." The emotion engine analyzes the user's input and presents additional customization options.

[1944] Server: Receives the entered customization information and stores it in a temporary database.

[1945] Image generation and confirmation

[1946] Server: Based on the saved customization information, the server sends an image generation request to the generative AI model. The generative AI model generates an image that reflects the customization options and sends it back to the server.

[1947] Terminal: The generated image is displayed to the user. The emotion engine analyzes the user's reactions in real time and makes suggestions to readjust the image if necessary.

[1948] Example: A customized image sent from the generative AI model is displayed on the device, and the user can review it. The emotion engine will make adjustments as necessary.

[1949] Confirmation of order details

[1950] User: Check the generated image and if satisfied, click the "Confirm Order" button. The emotion engine may also display recommendations based on the user's emotions.

[1951] Server: Receives order confirmation information and stores it in the database as official order data. Generates order information for related vendors and communicates it to the vendors.

[1952] Cooperation with vendors

[1953] Server: Sends order information via the vendor's API or email system and receives order confirmation from the vendor.

[1954] Example: Use a vendor API to send order information and get order confirmation from the vendor.

[1955] User Notice and Receipt

[1956] Server: Receives notification of product manufacturing completion from the supplier and sends notification of product manufacturing completion to the user via email or app notification.

[1957] On the device: The user receives a notification and checks the product delivery status on the delivery status page.

[1958] User: After receiving the product, press the confirmation button on the system.

[1959] Example: After a user receives a product, they press the "Confirm Receipt" button in the system.

[1960] Closing process and back margin payment

[1961] Server: Periodically extracts order and sales information from the database, calculates back margins, and issues invoices to related vendors.

[1962] Example: The server calculates the back margin based on the order data and sends an invoice to the relevant vendor.

[1963] Prompt Sentence Examples

[1964] "Design a system that allows users to customize items using an emotion engine and experience the ordering process. Required functionality includes login, product selection, customization options, image generation using generative AI, ordering, vendor integration, notifications, receipt confirmation, and closing."

[1965] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1966] Step 1: Access the system

[1967] Terminal: The user accesses the system using a smartphone or PC browser, or a dedicated app. The user enters their username and password on the login page and presses the login button.

[1968] Input: Username "user123", Password "password123"

[1969] Output: User dashboard

[1970] Specific operation: The user accesses "example.com / login", enters the username and password, and clicks the login button.

[1971] Server: Receives login information and retrieves the corresponding user information from the database. The retrieved information is compared with the entered information, and if authentication is successful, a user dashboard is generated and returned to the terminal.

[1972] Input: Login information (username and password)

[1973] Output: Authentication results and user dashboard

[1974] Specific operation: The server retrieves user information from the database, performs authentication, and if successful, creates and returns a user dashboard.

[1975] Step 2: Select product category

[1976] On the device: The user clicks the "Select a product" button on the dashboard, which takes them to the category selection page. The user then selects the desired category from the list of product categories provided.

[1977] Input: Category selection request

[1978] Output: Product category page

[1979] Specific behavior: The user clicks the "Choose a Product" button on the dashboard and selects the "Game Items" category.

[1980] Server: Based on the selected category, retrieves a list of related products from the database and returns it to the terminal, which then displays the product list to the user.

[1981] Input: Category selection information

[1982] Output: Product list

[1983] Specific operation: The server retrieves data in the "game items" category from the database, sends a product list to the user's terminal, and displays it.

[1984] Step 3: Configure customization options

[1985] Terminal: The user inputs customization options such as color, size, material, etc. The emotion engine analyzes the user's emotions in real time and suggests customization options based on the user's preferences and emotions.

[1986] Input: Customization option information

[1987] Output: Optimized customization options

[1988] What it does: The user sets the color of the "model sword" to "silver," the length to "50cm," and the material to "wood." The emotion engine uses these inputs to suggest additional customization options.

[1989] Server: Receives the entered customization information and stores it in a temporary database.

[1990] Input: User-entered customization information

[1991] Output: Save results to a temporary database

[1992] Specific operation: The server receives the customization information of "silver, 50cm, wood" and stores it in a temporary database.

[1993] Step 4: Image generation and verification

[1994] Server: Based on the saved customization information, the server sends an image generation request to the generative AI model. The generative AI model generates an image that reflects the customization options and sends it back to the server.

[1995] Input: Customization information

[1996] Output: Generated customized image

[1997] Specific operation: The server requests the generative AI model to generate an image of a "silver, 50 cm, wooden sword" and receives the generated image data.

[1998] Terminal: The generated image is displayed to the user. The emotion engine analyzes the user's reactions in real time and makes suggestions to readjust the image if necessary.

[1999] Input: Customized image data

[2000] Output: Optimized customized image

[2001] Specific operation: The image sent from the generative AI model is displayed on the device, and the emotion engine suggests readjustments based on the user's emotional response.

[2002] Step 5: Confirm your order

[2003] User: Check the generated image and if satisfied, click the "Confirm Order" button. The emotion engine may also display recommendations based on the user's emotions.

[2004] Input: Order confirmation request

[2005] Output: Order confirmation notification

[2006] Specific operation: The user checks the image and clicks the "Confirm Order" button.

[2007] Server: Receives order confirmation information and stores it in the database as official order data. Generates order information for related vendors and communicates it to the vendors.

[2008] Input: Order confirmation information

[2009] Output: Official order data and order information to vendors

[2010] Specific operation: The server saves the order information in a database and prepares to send the order information to the supplier.

[2011] Step 6: Contact a vendor

[2012] Server: Sends order information via the vendor's API or email system and receives order confirmation from the vendor.

[2013] Input: Order data

[2014] Output: Vendor order confirmation

[2015] Specific operation: The server sends the order information to the supplier and receives an order confirmation.

[2016] Step 7: User Notification and Acceptance

[2017] Server: Receives notification from the supplier that product manufacturing is complete and sends notification of product manufacturing completion to the user via email or app notification.

[2018] Input: Product manufacturing completion notification

[2019] Output: Product manufacturing completion notification to user

[2020] Specific operation: The server receives the product manufacturing completion information and sends a notification to the user.

[2021] On the device: The user receives a notification and checks the product delivery status on the delivery status page.

[2022] Input: Receive completion notification

[2023] Output: Display delivery status

[2024] What happens: A user checks the delivery status of a product online.

[2025] User: After receiving the product, press the confirmation button on the system.

[2026] Input: Receipt Acknowledgment Request

[2027] Output: Notification of receipt

[2028] Specific operation: The user receives the product and clicks the "Confirm receipt" button.

[2029] Step 8: Closing and margin payment

[2030] Server: Periodically extracts order and sales information from the database, calculates back margins, and issues invoices to related vendors.

[2031] Input: Order and sales data

[2032] Output: Back margin calculation results and invoice

[2033] Specific operation: The server calculates the back margin based on the order data for the specified period and sends an invoice to the relevant vendor.

[2034] (Application example 2)

[2035] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[2036] In current systems that allow users to receive items seen in virtual spaces or games as physical goods in the real world, the customization experience is complicated, making it difficult to achieve highly satisfying customization that reflects the user's emotions.Another issue is that the wide range of customization options makes it difficult for users to make the optimal choice.

[2037] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[2038] In this invention, the server includes: a means for allowing a user to select an item seen in a virtual space, game, or comic; a means for setting customization options such as color, size, and material for the item selected by the user; an emotion analysis means for analyzing the user's emotions in real time and suggesting customization options; a means for displaying an image generated based on the customization options; a means for the user to check the generated image and confirm the order details; and a means for coordinating the confirmed order details with related vendors to have the product manufactured and delivered. This enables a highly satisfying customization and ordering process that reflects the user's emotions.

[2039] A "virtual space" is a virtual environment generated by a computer.

[2040] A "game" is an interactive activity under prescribed rules for recreational or competitive purposes.

[2041] "Comics" is a form of publication that tells a story through pictures and text.

[2042] An "item" is a thing or element that exists in relation to a particular use or purpose.

[2043] "Customization options" are options that allow users to set the color, size, material, etc. of an item according to their preferences.

[2044] "Emotion analysis means" is a technology that analyzes emotions in real time from a user's facial expressions and behavior.

[2045] A "generated image" is a visual representation generated based on the customization options set by the user.

[2046] "Order details" refers to product specifications and order information confirmed by the user.

[2047] "Affiliated businesses" are companies or organizations that manufacture and deliver products based on user orders.

[2048] A "generative AI model" is an algorithm that uses artificial intelligence to generate images or text from data.

[2049] A "system" is a set of processes or devices consisting of multiple elements configured to achieve a specific purpose.

[2050] This invention provides a customization and ordering process that responds to the user's emotions by combining an emotion analysis engine with a system that allows users to customize items they see in virtual spaces, games, and comics and receive them as real products. The following describes an embodiment of the invention.

[2051] Hardware and software used

[2052] 1. Smartphone: Users use iOS or Android devices, and a dedicated application is installed on the smartphone.

[2053] 2. Emotion Analysis Engine: Affectiva SDK is used. This SDK analyzes the user's facial expression data and grasps their emotional state in real time.

[2054] 3. Generative AI model: OpenAI GPT and Gemini AI are used. OpenAI GPT performs natural language analysis and generation, and Gemini AI generates images based on customization options.

[2055] 4. Server: The server uses AWS (Amazon Web Services), which handles back-end processing, data storage, and vendor integration.

[2056] 5. Database: MySQL and DynamoDB are used to store customization information, order information, user information, etc.

[2057] 6. Notification system: Use Firebase Cloud Messaging (FCM) to send notifications to users.

[2058] 7. API communication: Data communication between systems is carried out using RESTful APIs.

[2059] Specific examples of the invention

[2060] A user accesses the system

[2061] Users access the system from a smartphone application and log in by entering their username and password. The server receives the login information, retrieves user information from the database, and performs authentication. If authentication is successful, it returns the user dashboard to the device.

[2062] The user selects the product they want

[2063] The user clicks the "Select a Product" button on the smartphone dashboard, which takes them to a category selection page. The user selects "Game Items" and then "Model Sword" from the list. The server retrieves a list of related products from the database based on the selected category and returns it to the device. The user then checks the product list on their smartphone.

[2064] Set customization options

[2065] Users input customization options such as color, size, and material. At this time, the sentiment analysis engine analyzes the user's emotions in real time and suggests customization options based on the user's preferences and emotions. For example, if a user sets the color of a "model sword" as silver, the length as 50 cm, and the material as wood, the sentiment analysis engine will suggest better options.

[2066] Image generation and confirmation using Gemini

[2067] The server sends an image generation request to Gemini AI, a generative AI model, based on the saved customization information. The image data of the customized product returned by Gemini AI is then received and returned to the device. The user can then view the generated image on their smartphone, and the sentiment analysis engine analyzes the user's reactions in real time, readjusting the image as necessary.

[2068] Confirmation of order details

[2069] The user checks the generated image and clicks the "Confirm Order" button to confirm the order. At this time, the sentiment analysis engine displays recommendations based on the user's emotions. The server receives the order confirmation information and stores it in the database as official order data. It also generates order information for related vendors.

[2070] Cooperation with vendors

[2071] The server sends the order information via the supplier's API or email system, receives an order confirmation from the supplier, and the supplier starts the manufacturing process based on the order information.

[2072] User Notice and Receipt

[2073] When the server receives a notification from the supplier that the product has been completed, it will notify the user by email or app notification. The user will receive the notification and check the product's delivery status on the delivery status page. When the user receives the product, they can press the receipt confirmation button on the system to complete the receipt.

[2074] Closing process and back margin payment

[2075] The server periodically extracts order and sales information from the database, calculates the back margin, and issues invoices to related vendors, allowing the system provider to earn revenue.

[2076] Prompt Sentence Examples

[2077] Below is an example prompt that can be input to a generative AI model to generate suggested customization options based on sentiment analysis.

[2078] text

[2079] The user selected the "Sword Model." Based on the user's facial expression data, their current emotional state is "Happy." Suggest the best customization options for this sword model based on the emotion of "Happy." Recommend color, material, and design changes as needed.

[2080] This prompt provides emotional data to the generative AI model, which then suggests appropriate customization options. This approach allows users to customize products in a way that best suits their emotions, resulting in a satisfying shopping experience.

[2081] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[2082] Step 1:

[2083] A user accesses the system and enters their login information.

[2084] Input: Username and Password

[2085] Specific operation: The user accesses the login screen of the smartphone app, enters their username and password, and presses the login button.

[2086] Data processing: The server receives the entered login information and retrieves user information from the database (MySQL).

[2087] Output: If authentication is successful, returns the user dashboard to the device.

[2088] Step 2:

[2089] The user selects the product they want.

[2090] Input: "Select a product" button and category selection (e.g., game items, sword model)

[2091] Specific Action: The user clicks the "Select a Product" button on the dashboard, goes to the category selection page, and then selects a specific item (e.g., "Model Sword") from the selected category.

[2092] Data processing: The server retrieves a list of related products from the database (DynamoDB) based on the selected category.

[2093] Output: The terminal displays the product list to the user.

[2094] Step 3:

[2095] Enter the customization options.

[2096] Input: Customization information such as item color, size, material, etc.

[2097] How it works: Users input customization options for the selected item, such as color, size, material, etc. At the same time, the emotion analysis engine (Affectiva SDK) analyzes the user's facial expression data in real time.

[2098] Data processing: The server sends the sentiment analysis data to OpenAI GPT, which then suggests customization options based on the user's emotions.

[2099] Output: Prints the suggested customization options to the terminal.

[2100] Step 4:

[2101] Check the generated image.

[2102] Input: Customization information and user emotion data

[2103] Specific operation: The user confirms the settings based on the customization options. After the settings are confirmed, the server sends an image generation request to Gemini AI based on the settings.

[2104] Data processing: Gemini AI creates a generated image based on the customization information and sends it back to the server.

[2105] Output: The server returns the generated image to the device, which displays it to the user. The sentiment analysis engine then analyzes the user's reaction again and adjusts the image if necessary.

[2106] Step 5:

[2107] Confirm the order details.

[2108] Input: Confirmed customizations and generated images

[2109] Specific operation: The user checks the generated image and clicks the "Confirm Order" button. At this time, the sentiment analysis engine displays recommendations based on the user's emotions.

[2110] Data processing: The server receives the order confirmation information, stores it in the database as official order data, and generates order information for related vendors.

[2111] Output: The order information is sent to the relevant supplier and the manufacturing process is initiated.

[2112] Step 6:

[2113] Product manufacturing and user notification.

[2114] Input: Production completion notice from supplier

[2115] Specific operation: The server receives notification from the relevant supplier that the product has been completed. Based on this information, it uses Firebase Cloud Messaging (FCM) to notify the user that the product has been completed.

[2116] Data processing: Generates notification information and sends it to the user's device.

[2117] Output: The user receives a notification on their smartphone that production is complete and checks the delivery status on the delivery status page.

[2118] Step 7:

[2119] Receipt and confirmation in the system.

[2120] Input: When the user receives the product, they press the "Confirm Receipt" button.

[2121] Specific operation: After receiving the product, the user presses the "Confirm receipt" button on the smartphone app, which causes the system to confirm receipt.

[2122] Data processing: The server records the received data and stores it in a database.

[2123] Output: The receipt confirmation information is saved in the database and the product receipt procedure is completed.

[2124] Step 8:

[2125] Regular data processing and back margin calculation.

[2126] Input: Order and sales information in the database

[2127] Specific operation: The server periodically extracts order information and sales information from the database, calculates the back margin based on this information, and issues invoices to related vendors.

[2128] Data processing: Analyze and calculate sales data and order information.

[2129] Output: Invoices are sent to the relevant vendors and back margin payments are processed.

[2130] As described above, by clearly indicating the specific operations, inputs, and outputs at each step, the user can smoothly complete the customization and ordering process and achieve a high level of satisfaction.

[2131] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the control target 443 to output the result of the specific processing. The microphone 238 acquires voice indicating a user input regarding the result of the specific processing. The control unit 46A transmits voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[2132] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[2133] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the robot 414.

[2134] The emotion identification model 59 as an emotion engine may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to an emotion map (see FIG. 9), which is a specific mapping. Similarly, the emotion identification model 59 may determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[2135] FIG. 9 is a diagram illustrating an emotion map 400 on which multiple emotions ...

Claims

1. A way for users to select items they see in virtual spaces, games, and comics; a means for the user to set customization options for the selected item, such as color, size, material, etc.; means for displaying an image generated based on the customization options; A means for a user to check the generated image and confirm the order details; A means of coordinating the confirmed order details with related vendors to manufacture and deliver the products; A system including:

2. A supplier cooperation means for receiving the order contents and starting the manufacturing process based on the order information; means for notifying the user that the product is complete; a means for calculating a back margin from sales information based on the order contents and invoicing the back margin to the related company; The system of claim 1 further comprising:

3. means for generating, using an AI generative model, an image generated based on the customization options; 10. The system of claim 1, further comprising means for a user to review the generated image.

Images