System

Abstract

To provide a system.SOLUTION: Provided is a system comprising: means for importing user information including at least either of an image of a user and requirements; means for recognizing the user's emotion using an emotion engine; prompt design means for making proposal pertaining to self coordination of the user on the basis of the user information and the user's emotion; and means for realizing the proposal in collaboration with external services.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] In modern society, there is a demand for products and services tailored to individual lifestyles and preferences. However, tailoring these recommendations to each individual requires advanced technology to process large amounts of information and make appropriate recommendations. In addition to making recommendations, a means is also needed to translate those recommendations into concrete action. A system that can solve these problems is needed. [Means for solving the problem]

[0005] The present invention provides a prompt design method that incorporates a user's image and requirements and suggests youthfulness, optimal products and purchasing methods, beauty and hairstyles, health maintenance methods, clothing, training methods, etc., a method for automatically linking with various services and production software, and a method for generating revenue by realizing optimal self-coordination. This makes it possible to provide suggestions tailored to each individual user and a method for turning those suggestions into concrete actions. [Brief explanation of the drawings]

[0006] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 2 is a sequence diagram showing a flow of processing in the data processing system according to the first embodiment of the first form example. [Figure 12]FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1 of Embodiment 1. [Figure 13] FIG. 10 is a sequence diagram showing a processing flow of a data processing system in a second embodiment of the second form example. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 of Embodiment Example 2. [Figure 15] FIG. 10 is a sequence diagram showing the flow of processing in a data processing system according to a third embodiment of the third embodiment. [Figure 16] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 3 of Embodiment 3. [Figure 17] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in the first embodiment of the first form example when an emotion engine is combined. [Figure 18] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1 of Form Example 1 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0007] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0008] First, the terms used in the following description will be explained.

[0009] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, the processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), an APU (Accelerated Processing Unit), or a TPU (TENSOR PROCESSING UNIT (registered trademark)).

[0010] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0011] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0012] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0013] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0014] [First embodiment]

[0015] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0016] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0017] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0018] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0019] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0020] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0021] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0022] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0023] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0024] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0025] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0026] Next, the specific processing by the specific processing unit 290 of the data processing device 12 will be described.

[0027] "Example 1"

[0028] The system of the present invention includes a means for capturing a user's image and requirements, a prompt design means, a means for automatically linking with various services and production software, and a means for realizing optimal self-coordination. The means for capturing a user's image and requirements acquires information from the user using a camera, microphone, input interface, etc. The prompt design means suggests youthfulness, optimal products and purchasing methods, beauty and hairstyles, health maintenance methods, clothing, training methods, etc. based on the acquired information. The means for automatically linking with various services and production software automatically purchases products and reserves services based on the suggested content. The means for realizing optimal self-coordination creates specific coordination based on the suggested content and presents it to the user.

[0029] "Example 2"

[0030] As a specific example, a user inputs an image of themselves and their desire to look youthful into the system. Based on this information, the prompt design means suggests youthful clothing, hairstyles, and health maintenance methods. When the user selects an outfit they like from the suggestions, the automatic linking means automatically connects to an online shop selling the selected outfit and completes the purchase process. Furthermore, the means for realizing optimal self-coordination creates a specific outfit based on the selected outfit and the user's image and presents it to the user.

[0031] The processing flow of each embodiment will be described below.

[0032] "Example 1"

[0033] Step 1: The user inputs their own image and requirements (e.g., a youthful appearance) into the system. This input is done using a camera, microphone, input interface, etc.

[0034] Step 2: The prompt design tool uses the input information to suggest youthful clothing, hairstyles, and health maintenance methods. These suggestions are based on the user's age, gender, body type, preferences, and other information.

[0035] Step 3: The user selects the outfit they like from the suggestions.

[0036] Step 4: The automatic linking means automatically connects to the online shop selling the selected clothing and carries out the purchase procedure.

[0037] Step 5: The means for realizing the optimal self-coordination creates a specific coordination based on the selected outfit and the user's image and presents it to the user.

[0038] "Example 2"

[0039] Step 1: The user inputs their image and requirements (e.g., health maintenance) into the system. This input is done using a camera, microphone, input interface, etc.

[0040] Step 2: The prompt design tool uses the input information to suggest healthy diets and exercise methods. These suggestions are based on the user's age, gender, body type, preferences, and other information.

[0041] Step 3: The user selects their preferred diet and exercise plan from the suggestions.

[0042] Step 4: An automatic linking means automatically makes reservations at restaurants that serve the selected meals and at training gyms that are suitable for the selected exercise method.

[0043] Step 5: The means for realizing optimal self-coordination creates a specific lifestyle based on the selected diet and exercise method and the user's image, and presents it to the user.

[0044] Example 1

[0045] Next, a description will be given of Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0046] Conventional self-coordination systems struggled to provide suggestions based on the individual requirements and characteristics of users, and lacked automatic coordination to translate the suggestions into concrete actions. As a result, users had to perform many manual operations, resulting in low convenience. Furthermore, the suggestions were not sufficiently detailed or visually presented, and there was a lack of ingenuity to make them easy for users to understand. This led to issues such as lower user satisfaction and a decrease in the frequency of system use.

[0047] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0048] In this invention, the server includes a means for capturing a user's image and requirements, a prompt design means for suggesting youthfulness, optimal products and purchasing methods, beauty and hairstyles, health maintenance methods, clothing, and training methods, a means for automatically linking with various services and production software, a means for generating revenue by realizing optimal self-coordination plans, a means for analyzing acquired image data using facial recognition technology, a means for converting voice data into text using speech recognition technology, a means for generating prompts using a generative AI model, a means for creating specific self-coordination plans based on the suggested content, and a means for presenting the generated self-coordination plans to the user. This enables suggestions based on the user's individual requirements and characteristics, realizes automatic linkage to translate the suggested content into concrete actions, and improves user convenience. Furthermore, the concreteness and visual presentation of the suggested content improves user understanding, improving user satisfaction and increasing system usage frequency.

[0049] "Means for capturing user images and requirements" refers to devices and software for obtaining image data, audio data, and text data from users using a camera, microphone, input interface, etc.

[0050] "Prompt design tools" are algorithms and software that use acquired user information to suggest youthfulness, optimal products and purchasing methods, beauty and hairstyles, health maintenance methods, clothing, training methods, etc.

[0051] "Means for realizing automatic integration with each service and production software" refers to an interface or API that automatically connects with external services such as online shops and reservation systems based on the proposed content, allowing products to be purchased or services to be reserved.

[0052] "Means for generating revenue by realizing optimal self-coordination" refers to a method or system for generating revenue by creating specific self-coordination based on proposed content and presenting it to users, thereby encouraging them to purchase products or use services.

[0053] "Means of analysis using facial recognition technology" refers to algorithms or software for extracting a user's age, gender, facial features, etc. from the acquired image data.

[0054] "Means of converting voice data into text using speech recognition technology" refers to algorithms or software that convert acquired voice data into text data and understand the user's requirements.

[0055] "Means for generating prompts using a generative AI model" refers to an artificial intelligence model or software that generates prompts containing suggestions based on the analysis results in accordance with the user's requirements.

[0056] The "means for creating specific self-coordination" refers to image processing technology and software that synthesizes clothing and hairstyles with the user's image based on the proposed content, generating a finished image.

[0057] The "means for presenting the generated self-coordinated image to the user" refers to an interface or software for transmitting the generated self-coordinated image to the user's terminal and displaying it.

[0058] This invention is a system that includes a means for capturing a user's image and requirements, a prompt design means, a means for automatically linking with various services and production software, and a means for realizing optimal self-coordination. Specific embodiments of this system are described below.

[0059] Retrieving User Information

[0060] Users access the system using a device (such as a smartphone or PC). They take a photo of themselves using the device's camera and input their requirements by voice using the microphone. Requirements can also be input as text using a keyboard or touch screen. For example, a user can take a photo of themselves using the smartphone's camera and say, "I'd like suggestions for a youthful style" using the microphone.

[0061] Sending data

[0062] The device sends the acquired image data, audio data, and text data to the server via an internet connection, such as Wi-Fi or mobile data.

[0063] Data analysis

[0064] The server analyzes the received data. For image data, it uses facial recognition technology (e.g., OpenCV) to extract the user's age, gender, and facial features. For voice data, it uses speech recognition technology (e.g., Google® Speech-to-Text API) to convert it into text and understand the user's requirements. For text data, it uses natural language processing technology (e.g., spaCy) to analyze the requirements.

[0065] Generate prompt statement

[0066] The server generates a prompt sentence using a generative AI model (e.g., GPT-4 (registered trademark)) based on the analysis results. For example, if a user inputs "Please suggest a youthful style," the server generates the prompt sentence "Please suggest a youthful casual style for a woman in her 30s."

[0067] Service collaboration

[0068] The server then connects to various services and production software based on the generated prompt text. For example, it uses an online shop's API to search for suggested outfits and proceed with the purchase. It also connects with a hair salon's reservation system to make a reservation for a hairstyle.

[0069] Coordination details

[0070] The server then creates a specific self-coordinated outfit based on the suggestions. For example, it uses image processing software to combine the suggested clothing and hairstyle with the user's photo to create a finished image.

[0071] Presentation of results

[0072] The server sends the generated self-coordinated image to the terminal, which displays the results to the user. The user can then review the proposed coordinated outfit and make purchases or reservations as necessary.

[0073] In this way, users can easily find the perfect outfit for themselves. A concrete example of a prompt sentence is, "Please suggest a youthful, casual style for a woman in her 30s." This system makes suggestions based on the user's individual requirements and characteristics, and automatically links the suggestions to concrete actions. This improves user convenience and enhances user satisfaction by making the suggestions more concrete and visually presenting them.

[0074] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0075] Step 1:

[0076] Users access the system using a terminal. They take their own photo with the terminal's camera and input their requirements by voice using the microphone. They can also input requirements in text using a keyboard or touch screen. Input data can be image data, voice data, or text data.

[0077] Step 2:

[0078] The device transmits the acquired image data, voice data, and text data to the server via an Internet connection. The input is the data acquired from the user, and the output is the data sent to the server.

[0079] Step 3:

[0080] The server analyzes the received image data using facial recognition technology (e.g., OpenCV) to extract the user's age, gender, and facial features. The input is image data, and the output is the user's feature data as the analysis result.

[0081] Step 4:

[0082] The server converts the received voice data into text using speech recognition technology (e.g., Google Speech-to-Text API) to understand the user's requirements. The input is voice data, and the output is text data.

[0083] Step 5:

[0084] The server analyzes the received text data using natural language processing technology (e.g., spaCy) to understand the user's requirements. The input is text data, and the output is requirement data as the analysis result.

[0085] Step 6:

[0086] The server generates a prompt sentence using a generative AI model (e.g., GPT-4) based on the analysis results. For example, if a user inputs "Please suggest a youthful style," the server generates the prompt sentence "Please suggest a youthful casual style for women in their 30s." The input is the analysis result, and the output is the prompt sentence.

[0087] Step 7:

[0088] The server interacts with various services and production software based on the generated prompt text. For example, it uses an online shop's API to search for suggested outfits and proceed with the purchase process. It also interacts with a hair salon's reservation system to make a hairstyle reservation. The input is the prompt text, and the output is the result of the service interaction.

[0089] Step 8:

[0090] The server creates a specific self-coordinated outfit based on the proposed content. For example, it uses image processing software to combine the suggested clothing and hairstyle with the user's photo to generate a finished image. The input is the proposed content and the user's photo, and the output is the finished image.

[0091] Step 9:

[0092] The server sends the generated self-coordinated image to the terminal, which displays the result to the user. The input is the completed image, and the output is the display to the user. The user can review the proposed coordinated image and make purchases or reservations as necessary.

[0093] (Application example 1)

[0094] Next, a description will be given of Application Example 1 of Embodiment Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0095] Conventional self-coordination systems have the problem that it is difficult for users to find the suggested products in physical stores, resulting in an inefficient shopping experience. Additionally, while there are systems that suggest optimal outfits based on the user's images and requirements, they lack the functionality to provide navigation information within physical stores, making it difficult for users to quickly find the suggested products.

[0096] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means. In this invention, the server includes a means for capturing a user's image and requirements, a prompt design means for suggesting youthfulness, optimal products and purchasing methods, beauty and hairstyles, health maintenance methods, clothing, and training methods, etc., a means for automatically linking with each service and production software, a means for generating revenue by realizing optimal self-coordination, and a means for providing navigation information for finding suggested products in a physical store. This allows the user to quickly find the suggested products in the physical store, making the shopping experience efficient and enjoyable.

[0097] "Means for capturing user images and requirements" refers to means for obtaining information from the user using a camera, microphone, input interface, etc.

[0098] "Prompt design methods" are methods that use acquired information to suggest youthfulness, optimal products and purchasing methods, beauty and hairstyles, health maintenance methods, clothing, training methods, etc.

[0099] "Means for realizing automatic integration with each service and production software" refers to a means for automatically purchasing products or reserving services based on the proposed content.

[0100] "A means of generating revenue by realizing optimal self-coordination" is a means of generating revenue by creating specific coordination based on the proposed content and presenting it to users.

[0101] The "means for providing navigation information for finding suggested products in a physical store" refers to a means for navigating so that suggested products can be easily found in a physical store.

[0102] As an embodiment of the present invention, the following system is constructed.

[0103] First, the smartphone's camera, microphone, and input interface are used to capture the user's image and requirements. The user takes a picture of themselves using the smartphone's camera and imports it into the app, which then inputs the user's image and requirements into the system.

[0104] Next, as a prompt design method, based on the acquired user image and requirements, the system suggests youthfulness, optimal products and purchasing methods, beauty and hairstyles, health maintenance methods, clothing, training methods, etc. For this, a generative AI model using Keras and TENSORFLOW (registered trademark) is used. Specifically, the user's image is input into the AI ​​model, and a prompt is generated that suggests the optimal outfit.

[0105] Furthermore, as a means of achieving automatic integration with various services and production software, products can be purchased or services can be reserved automatically based on the proposed content, allowing users to easily purchase the proposed products or reserve the services.

[0106] In addition, as a means of generating revenue by realizing optimal self-coordination, specific coordinations are created based on the proposed content and presented to users, allowing them to actually try on the suggested coordinations.

[0107] Finally, as a means for providing navigation information for finding the suggested product in a physical store, navigation is provided so that the suggested product can be easily found in the physical store. Specifically, based on the suggested product, information is provided for navigating to where the product is located in the physical store.

[0108] For example, a user takes a picture of themselves with their smartphone camera and imports it into the app. The app then suggests, "The recommended outfit is a shirt, pants, and shoes." In a physical store, the app navigates to, "The shirt is in Aisle 3, the pants are in Aisle 5, and the shoes are in Aisle 7."

[0109] Examples of prompts include, "Based on the user's image, suggest the best fashion outfit" and "Based on the suggested outfit, provide navigation information to find the product in a physical store."

[0110] In this way, users can quickly find suggested products in physical stores, making their shopping experience efficient and enjoyable.

[0111] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0112] Step 1:

[0113] The user takes a picture of themselves with the smartphone camera and imports it into the app. The input is the user's image, and the output is image data. Specifically, the smartphone's camera app is launched and the user takes a picture of themselves. The captured image is saved in the app.

[0114] Step 2:

[0115] The image data captured by the device is input into a generative AI model to generate a prompt. The input is image data, and the output is a prompt. Specifically, the image data is input into a generative AI model using Keras and TensorFlow, and the model analyzes the image and generates a prompt that suggests the optimal outfit.

[0116] Step 3:

[0117] Based on the generated prompt, the server suggests the most suitable products and purchasing methods for the user, beauty and hairstyles, health maintenance methods, clothing, training methods, etc. The input is the prompt, and the output is the suggestions. Specific operations include analyzing the prompt and suggesting specific products and services to the user.

[0118] Step 4:

[0119] The server automatically connects with each service and production software based on the proposed content. The input is the proposed content, and the output is product purchase or service reservation information. Specifically, it calls an API to automatically purchase or reserve the proposed product or service.

[0120] Step 5:

[0121] The server creates the optimal self-coordinated outfit and presents it to the user. The input is the proposed content, and the output is specific coordinated information. Specifically, the server presents the specific coordinated outfit to the user in the form of images and text based on the proposed content.

[0122] Step 6:

[0123] The terminal provides navigation information to find the suggested product in the physical store. The input is the suggested product information, and the output is navigation information. Specifically, based on the suggested product, it provides information to navigate to the location of the product in the physical store. The user can navigate around the store and find the suggested product while looking at the smartphone screen.

[0124] Example 2

[0125] Next, a description will be given of Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0126] In conventional systems, even if users input their own images and requirements, it was difficult for them to receive appropriate suggestions, and the purchasing process for the suggested products and services was cumbersome. Furthermore, there was a lack of means for users to specifically check the clothes and hairstyles they selected, making it difficult to achieve optimal self-coordination.

[0127] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0128] In this invention, the server includes means for importing a user's image and requirements, a prompt design means for suggesting youthfulness, optimal products and purchasing methods, beauty and hairstyles, health maintenance methods, clothing, training methods, etc., a means for automatically linking with each service and production software, a means for generating monetization by realizing optimal self-coordination, a means for generating suggestions using a generative AI model, a means for connecting to an online shop based on the user's selection and completing the purchase process, a means for generating coordinated images based on the selected clothing and the user's image, and a means for presenting the generated coordinated images to the user. This allows the user to easily receive appropriate suggestions, smoothly complete the purchase process for the selected products and services, and further view specific coordinated images.

[0129] "Means for capturing user images and requirements" refers to a function for receiving image data and requirement information provided by the user and converting them into a format that can be processed within the system.

[0130] The "prompt design means" is a function that generates prompt sentences to be input into the generative AI model based on information provided by the user.

[0131] "Means for realizing automatic collaboration with each service and production software" refers to a function that enables the system to automatically collaborate with external services and software to exchange and process data.

[0132] "A means of generating revenue by realizing optimal self-coordination" is a function that shows specific coordinated outfits suggested to users and generates revenue as a result.

[0133] A "means for generating proposals using a generative AI model" is a function that uses a generative AI model to automatically generate proposals based on user requirements.

[0134] "Means of connecting to an online store based on the user's selection and completing the purchase process" is a function that automatically connects to an online store that sells the product selected by the user and completes the purchase process.

[0135] The "means for generating a coordinated image based on selected clothing and the user's image" is a function for combining the clothing selected by the user with the user's image to generate a specific coordinated image.

[0136] The "means for presenting the generated coordinated image to the user" is a function for displaying the generated coordinated image on the user's terminal so that the user can check it.

[0137] This invention is a system that allows users to input their own image and requirements, and then suggests youthful clothing, hairstyles, health maintenance methods, etc., and generates specific coordinated images. This system operates in cooperation with the server, terminal, and user.

[0138] First, the user uses a device such as a smartphone or PC to take or select a photo of themselves and input the requirement of "looking youthful." The device then sends the image and requirements entered by the user to the server. Specifically, the data is sent using an HTTP POST request.

[0139] The server generates a prompt sentence to be input to the generative AI model based on the received image and requirements. For example, it generates a prompt sentence such as "Please suggest youthful clothing, hairstyles, and health maintenance methods for this user." The server uses a generative AI model (e.g., GPT-4 from OpenAI (registered trademark)) to generate suggestions based on the prompt sentence. The suggestions include youthful clothing, hairstyles, and health maintenance methods.

[0140] The generated suggestions are sent from the server to the device. The user selects their favorite outfit and hairstyle from the suggestion list displayed on the device. For example, they select "white shirt and denim pants." The device then sends the user's selected data to the server. Specifically, it sends the data again using an HTTP POST request.

[0141] The server connects to the API of the online shop selling the selected outfit and automatically completes the purchase process. Specifically, it adds the item to the cart and proceeds with the purchase. The server then uses image processing software to generate a coordinated image based on the selected outfit and the user's image.

[0142] The generated outfit image is sent from the server to the device. The user can check the outfit image displayed on the device and visually confirm the specific outfit. For example, an image of the user wearing a white shirt and denim pants is displayed on the smartphone screen.

[0143] This system allows users to easily receive appropriate suggestions, smoothly complete the purchasing process for selected products and services, and even view specific outfit images.

[0144] As a concrete example, consider the case where a user takes a photo of themselves with their smartphone and inputs, "I'd like suggestions for youthful clothing." The server uses a generative AI model to suggest "white shirt and denim pants, short hairstyle, daily jogging." The user selects "white shirt and denim pants," and the server completes the purchase process using the online shop's API. The server combines the white shirt and denim pants with the user's image to generate an outfit image. An image of the user wearing the white shirt and denim pants is displayed on the user's smartphone.

[0145] Examples of prompts include, "Suggest youthful clothing, hairstyles, and health maintenance methods for this user" and "Create the perfect outfit based on the user's image."

[0146] In this way, users can simply enter their own image and requirements to receive suggestions for youthful clothing, hairstyles, and health maintenance methods, and can also view specific outfit images.

[0147] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0148] Step 1:

[0149] The user inputs the image and requirements.

[0150] Users use a smartphone or PC to take or select a photo of themselves and input their desired "youthfulness" criteria. The input data consists of an image file and text data.

[0151] Step 2:

[0152] The terminal sends the input data to the server.

[0153] The terminal sends the image file and text data entered by the user to the server via an HTTP POST request. The input is the user's image and requirements, and the output is data sent to the server.

[0154] Step 3:

[0155] The server generates a prompt.

[0156] The server analyzes the received image file and text data and generates a prompt such as, "Please suggest youthful clothing, hairstyles, and health maintenance methods for this user." The input is the image file and text data, and the output is the prompt.

[0157] Step 4:

[0158] The server generates suggestions using a generative AI model.

[0159] The server inputs a prompt into a generative AI model (e.g., OpenAI's GPT-4) and generates suggestions for youthful clothing, hairstyles, and health maintenance methods. The input is the prompt, and the output is the suggestion data.

[0160] Step 5:

[0161] The server sends the proposal to the device.

[0162] The server sends the generated proposal to the terminal in JSON format. The input is the proposal data, and the output is the data sent to the terminal.

[0163] Step 6:

[0164] The user selects from the suggestions.

[0165] The user selects the clothes and hairstyle they like from the suggestion list displayed on the terminal. For example, they select "white shirt and denim pants." The input is the suggestion list, and the output is the selected data.

[0166] Step 7:

[0167] The terminal transmits the selection data to the server.

[0168] The terminal sends the data of the user's selection, "white shirt and denim pants," to the server via an HTTP POST request. The input is the selected data, and the output is the data sent to the server.

[0169] Step 8:

[0170] The server connects to the online shop and completes the purchase.

[0171] The server connects to the API of the online shop selling the selected clothing and automatically completes the purchase process. Specifically, it adds the item to the cart and proceeds with the purchase. The input is the selected data, and the output is the completion of the purchase process.

[0172] Step 9:

[0173] The server generates a coordinated image.

[0174] The server generates a coordinated image using image processing software based on the selected outfit and the user's image. The input is the user's image and the selected outfit, and the output is the coordinated image.

[0175] Step 10:

[0176] The server transmits the coordinated image to the terminal.

[0177] The server sends the generated coordinated image to the terminal as an HTTP response. The input is the coordinated image, and the output is data sent to the terminal.

[0178] Step 11:

[0179] The user checks the coordinated outfit image.

[0180] The user checks the outfit image displayed on the device and visually confirms the specific outfit. For example, an image of the user wearing a white shirt and denim pants is displayed on the smartphone screen. The input is the outfit image, and the output is the user's confirmation.

[0181] (Application example 2)

[0182] Next, a description will be given of Application Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0183] In conventional online shopping and styling suggestion systems, even when users input their own images and requirements, the suggested products and styles often did not meet the user's expectations. Furthermore, the process of actually purchasing the suggested products was cumbersome, and there was a lack of mechanisms for users to easily make purchases. Furthermore, users were unable to try on the selected outfits, often resulting in dissatisfaction after purchase. To solve these problems, a system is needed that suggests optimal products and styles based on the user's images and requirements, facilitates the purchasing process, and provides a virtual try-on function.

[0184] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0185] In this invention, the server includes means for capturing a user's image and requirements, a prompt design means for suggesting youthfulness, optimal products and purchasing methods, beauty and hairstyles, health maintenance methods, clothing, training methods, etc., a means for automatically linking with each service and production software, a means for generating revenue by realizing optimal self-coordination, a means for creating specific coordination based on the user's image and selected clothing and presenting it to the user through a virtual try-on function, and a means for generating prompts using a generative AI model.This allows the user to receive suggestions for optimal products and styles based on their image and requirements, easily complete the purchase process, and try on the selected clothing through the virtual try-on function.

[0186] "Means for capturing user images and requirements" refers to an interface or device that allows a user to input their own image and desired requirements into the system.

[0187] "Prompt design tools" are algorithms and software that suggest youthfulness, optimal products, beauty and hairstyles, health maintenance methods, clothing, training methods, etc. based on the user's images and requirements.

[0188] "Automatic linking means" refers to a system that automatically connects the product or service selected by the user to the relevant online shop or service provider to complete the purchase procedure or make a reservation.

[0189] The "means for realizing self-coordination" refers to software or algorithms that create specific coordination based on the user's image and selected clothing and present it to the user.

[0190] The "virtual try-on feature" is a technology that allows users to see the outfit they have selected superimposed on their own image, making it appear as if they are trying it on.

[0191] A "generative AI model" is an artificial intelligence model that generates prompts based on the user's images and requirements and makes optimal suggestions.

[0192] The following system configuration is used as an embodiment of the present invention.

[0193] The server includes a means for capturing user images and requirements, a prompt design means, an automatic linking means, a means for realizing self-coordination, a virtual try-on function, and a generative AI model.

[0194] Users use devices such as smartphones or head-mounted displays to take a photo of themselves and input their desired youthful appearance, which is then sent to the server.

[0195] The server receives the user's image and requirements using a means for capturing the user's image and requirements, and then uses a prompt design means to suggest youthful clothing, hairstyles, health maintenance methods, etc. based on the user's image and requirements. The prompt design means uses a generative AI model to generate prompts and make optimal suggestions.

[0196] When a user selects an outfit they like from the suggestions, an automatic linking means is activated, automatically connecting them to the online shop selling the selected outfit and carrying out the purchase procedure, allowing the user to easily purchase the item.

[0197] Furthermore, a means for realizing self-coordination is used to create a specific outfit based on the user's image and the selected outfit. This outfit is presented to the user through a virtual try-on function. The user can see the selected outfit superimposed on their own image, allowing them to check it as if they were trying it on.

[0198] For example, if a user takes a photo of themselves with their smartphone and enters the requirement for "youthful clothing," the server will suggest a youthful casual jacket. When the user selects this jacket, the server automatically connects to the online shop and completes the purchase. The server then combines the user's image with an image of the selected jacket to create a specific outfit and presents it to the user through a virtual try-on function.

[0199] An example prompt is:

[0200] Enter the user image: path / to / user_image.jpg

[0201] Enter your youthfulness requirements: Youthful clothing

[0202] Choose the suggested outfit: Youthful casual jacket

[0203] This way, users can easily find, shop and try on youthful styles.

[0204] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0205] Step 1:

[0206] Users use a device such as a smartphone or head-mounted display to take a photo of themselves and input their desired youthful appearance.

[0207] Input: User image, youthfulness requirement

[0208] Output: User image data, requirement data

[0209] Specific behavior: Use the device's camera function to take a picture and enter the requirements in the text input field.

[0210] Step 2:

[0211] The terminal transmits the user's image data and requirement data to the server.

[0212] Input: User image data, requirement data

[0213] Output: Image data and requirements data sent to the server

[0214] Specific operation: Using the terminal's communication function, image data and requirement data are uploaded to the server.

[0215] Step 3:

[0216] The server receives the user's image data and requirement data, and proposes youthful clothing, hairstyles, health maintenance methods, etc. using prompt design means.

[0217] Input: User image data, requirement data

[0218] Output: A list of suggested youthful clothing, hairstyles, and health maintenance methods

[0219] What it does: It uses a generative AI model to generate prompts and make optimal suggestions based on the user's image and requirements.

[0220] Step 4:

[0221] The user selects the outfit they like from the suggestions.

[0222] Input: A list of suggested outfits

[0223] Output: Selected outfit

[0224] What it does: Use the device interface to select from a list of suggested outfits.

[0225] Step 5:

[0226] The server uses an automatic linking means to automatically connect to an online shop selling the selected clothing and carry out the purchase procedure.

[0227] Input: selected outfit

[0228] Output: Purchase completion notification

[0229] Specific operation: The server uses the online shop's API to automatically complete the purchase process for the selected clothing.

[0230] Step 6:

[0231] The server uses a means for realizing self-coordination to create a specific coordination based on the user's image and selected clothes.

[0232] Input: User image data, selected outfit

[0233] Output: Specific coordinated image

[0234] Specific operation: Using an image processing algorithm, the system combines the user's image with the image of the selected outfit to generate an outfit image.

[0235] Step 7:

[0236] The server presents specific outfit images to the user through a virtual try-on function.

[0237] Input: Specific coordinated image

[0238] Output: Coordinate image presented to the user

[0239] Specific operation: Coordinated images are displayed on the device display, allowing users to check the outfits as if they were trying them on.

[0240] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0241] "Example 1"

[0242] One embodiment of the present invention is a system that incorporates an emotion engine. This system uses a camera, microphone, input interface, etc. as a means of capturing a user's image and requirements. Furthermore, this capturing means is equipped with an emotion engine, which can recognize emotions from the user's facial expressions and tone of voice. For example, if the user is smiling, it is recognized as "joy," if they frown, it is "anger," and if their voice is low-pitched, it is "sadness."

[0243] "Example 2"

[0244] Next, based on the captured user information and the emotions recognized by the emotion engine, the prompt design tool suggests youthfulness, optimal products and purchasing methods, beauty and hairstyles, health maintenance methods, clothing, training methods, etc. For example, if the user expresses the emotion of "joy," it will suggest bright-colored clothing that the user would like and exercise methods that match their happy mood.

[0245] "Example 4"

[0246] Furthermore, the automatic linking means automatically purchases products or reserves services based on the user's selection and emotions. For example, if the user expresses "joy," the system automatically connects to an online shop selling brightly colored clothing selected by the user and completes the purchase. If the user expresses "sad," the system automatically reserves a relaxation service selected by the user.

[0247] "Example 5"

[0248] Finally, the means for realizing optimal self-coordination creates specific coordination based on the selected products and services, the user's image, and the emotions recognized by the emotion engine, and presents it to the user. For example, for a user who expresses the emotion of "joy," it will present an outfit that combines brightly colored clothing with an exercise method that matches the user's happy mood.

[0249] The processing flow of each embodiment will be described below.

[0250] "Example 1"

[0251] Step 1: The system captures the user's image and requirements using a camera, microphone, input interface, etc.

[0252] Step 2: The emotion engine built into the capture device recognizes emotions from the user's facial expressions and tone of voice.

[0253] "Example 2"

[0254] Step 1: The prompt design tool suggests youthfulness, optimal products and purchasing methods, beauty and hairstyles, health maintenance methods, clothing, training methods, etc. based on the captured user information and emotions recognized by the emotion engine.

[0255] Step 2: If the user expresses the emotion "joy," suggest bright-colored clothing that the user might like, or exercise methods that match their happy mood.

[0256] "Example 4"

[0257] Step 1: The automatic interaction means automatically purchases products or reserves services based on the user's choices and emotions.

[0258] Step 2: If the user shows the emotion "joy," the system automatically connects them to an online shop selling the brightly colored clothing they selected and completes the purchase. Step 3: If the user shows the emotion "sad," the system automatically makes a reservation for the relaxation service they selected.

[0259] "Example 5"

[0260] Step 1: The means for realizing the optimal self-coordination is to create a specific coordination based on the selected product or service, the user's image, and the emotions recognized by the emotion engine.

[0261] Step 2: For users who show the emotion of "joy," we suggest outfits that combine brightly colored clothing with exercise methods that match their happy mood.

[0262] Example 1

[0263] Next, a description will be given of Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0264] Conventional self-coordination systems struggled to provide proposals that fully reflected the individual needs and feelings of users, and lacked a means to visualize the proposed content in concrete terms. This made the process from the user understanding the proposals to actually taking action complicated, making it difficult to improve the user experience. Furthermore, linking with various services and production software was often done manually, making efficient operation difficult.

[0265] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0266] In this invention, the server includes a means for capturing a user's image and requirements, a prompt design means for suggesting youthfulness, optimal products and purchasing methods, beauty and hairstyles, health maintenance methods, clothing, and training methods, a means for automatically linking with various services and production software, a means for generating revenue by realizing optimal self-coordination, a means including an emotion engine for recognizing emotions from the user's facial expressions and tone of voice, a means for generating prompts using a generative AI model, and a means for generating virtual try-on images based on the suggested content. This enables suggestions that reflect the user's individual requests and emotions, and the concrete visualization of the suggested content improves the user experience. Furthermore, automatic linking with various services and production software enables efficient operation.

[0267] "Means for capturing user images and requirements" refers to means for obtaining information from the user using a camera, microphone, or input interface.

[0268] "Prompt design methods" are methods that use acquired information to suggest youthfulness, optimal products and purchasing methods, beauty and hairstyles, health maintenance methods, clothing, training methods, etc.

[0269] "Means for realizing automatic integration with each service and production software" refers to a means for automatically purchasing products or reserving services based on the proposed content.

[0270] The "means for realizing optimal self-coordination" is a means for creating a specific coordination based on the proposed content and presenting it to the user.

[0271] "Means including an emotion engine" refers to means that has the function of recognizing emotions from the user's facial expressions and tone of voice.

[0272] "Means for generating prompts using a generative AI model" refers to means for generating prompts using a generative AI model based on the analysis results.

[0273] The "means for generating a virtual try-on image" is a means for generating an image in which the user virtually tries on clothes based on the proposed content.

[0274] The system of the present invention includes a means for capturing a user's image and requirements, a prompt design means, a means for automatically linking with each service and production software, a means for realizing optimal self-coordination, a means including an emotion engine, a means for generating prompts using a generative AI model, and a means for generating virtual try-on images.

[0275] A way to capture user images and requirements

[0276] Users take a photo of themselves using the camera on their smartphone or computer, and then speak their request into the microphone. This data is temporarily stored on the device and then sent to the server.

[0277] Sending and storing information

[0278] The device sends the acquired image data and audio data to the server. The server stores the received data in a database. For example, image data is stored in the "user_images" table, and audio data is stored in the "user_requests" table.

[0279] Analysis of information

[0280] The server uses image analysis algorithms to estimate the user's age and gender. For example, it uses OpenCV for facial recognition and an age estimation model to estimate age. It also uses voice recognition software (e.g., Google Cloud Speech-to-Text) to convert voice data into text. It also uses an emotion engine (e.g., Microsoft® Azure® Emotion API) to recognize emotions from the user's facial expressions and tone of voice.

[0281] Prompt Generation

[0282] The server generates prompts using a generative AI model (e.g., GPT-4) based on the analysis results. For example, it generates a prompt such as, "For a smiling woman in her late 30s, suggest casual clothing that emphasizes youthfulness."

[0283] Service collaboration

[0284] The server then connects with various services based on the generated prompts. For example, it uses an online shop's API to search for suggested outfits and complete the purchase process. It also connects with a hair salon's reservation system to book the most suitable salon for the user.

[0285] Coordination details

[0286] The server then creates a specific self-coordinated outfit based on the suggested content, for example, by generating an image of the user virtually trying on the suggested outfit. For this purpose, it uses image processing software.

[0287] Presentation of results

[0288] The server then sends the generated virtual try-on images and the results of the purchase process to the device. The user can then check the outfit on their smartphone or computer screen. For example, the device will display a notification, and when the user opens the app, the virtual try-on images and a link to purchase will be displayed.

[0289] Examples of specific examples and prompts

[0290] As a concrete example, consider the case where a user takes a photo of themselves with their smartphone camera and then voice-inputs, "Please suggest some clothes that will make me look younger." The system estimates the user's age and gender, and uses an emotion engine to recognize that the user is smiling. Based on this, the system suggests casual clothes that emphasize youthfulness, automatically searches for the clothes in an online shop, and completes the purchase process. It also generates virtual try-on images and presents them to the user.

[0291] Example prompt sentence:

[0292] "Based on the user's image and voice input, please suggest casual clothing that emphasizes youthfulness. The user is smiling, in their late 30s, and female. Search for the suggested outfits in an online shop, complete the purchase process, and generate and present a virtual try-on image."

[0293] In this way, the system of the present invention collects and analyzes user information, generates prompts, and works with various services to provide optimal self-coordination.

[0294] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0295] Step 1:

[0296] Users take a photo of themselves using the camera on their smartphone or computer, and then input their request by voice using the microphone. The input data consists of image data and voice data, which are temporarily stored on the device.

[0297] Step 2:

[0298] The device sends the saved image data and audio data to the server. The server saves the received data in a database. For example, image data is saved in the "user_images" table, and audio data is saved in the "user_requests" table. The input is image data and audio data, and the output is saved in the database.

[0299] Step 3:

[0300] The server uses an image analysis algorithm to estimate the user's age and gender. For example, it uses OpenCV for facial recognition and estimates age using an age estimation model. The input is image data, and the output is the estimated age and gender. It also uses speech recognition software (e.g., Google Cloud Speech-to-Text) to convert voice data into text. The input is voice data, and the output is text data. It then uses an emotion engine (e.g., Microsoft Azure Emotion API) to recognize emotions from the user's facial expressions and tone of voice. The input is image data and voice data, and the output is the recognized emotion.

[0301] Step 4:

[0302] The server generates a prompt using a generative AI model (e.g., GPT-4) based on the analysis results. For example, it generates a prompt such as "Suggest casual clothing that emphasizes youthfulness to a smiling woman in her late 30s." The input is the estimated age, gender, emotion, and text data, and the output is the generated prompt.

[0303] Step 5:

[0304] The server interacts with various services based on the generated prompt. For example, it uses an online shop's API to search for suggested outfits and then completes the purchase process. The input is the generated prompt, and the output is the completion of the purchase process. It also interacts with a hair salon reservation system to make a reservation at the most suitable hair salon for the user. The input is the generated prompt, and the output is the completion of the reservation.

[0305] Step 6:

[0306] The server creates a specific self-coordinated outfit based on the suggested content. For example, it generates an image of the user virtually trying on the suggested outfit. For this purpose, it uses image processing software. The input is the user's image data and the suggested outfit data, and the output is a virtual try-on image.

[0307] Step 7:

[0308] The server sends the generated virtual try-on images and the results of the purchase process to the device. The user can then check the outfit on their smartphone or computer screen. The input is the virtual try-on images and the results of the purchase process, and the output is what is presented to the user. For example, the device displays a notification, and when the user opens the app, the virtual try-on images and a link to purchase are displayed.

[0309] (Application example 1)

[0310] Next, a description will be given of Application Example 1 of Embodiment Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0311] Conventional self-coordination systems were unable to provide suggestions that took user emotions into consideration, and were therefore unable to fully increase user satisfaction. Furthermore, they lacked the ability to purchase or reserve suggested items on the spot, resulting in low user convenience. This limited the user's shopping experience and missed monetization opportunities.

[0312] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0313] In this invention, the server includes a means for capturing a user's image and requirements, a prompt design means for suggesting youthfulness, optimal products and purchasing methods, beauty and hairstyles, health maintenance methods, clothing, training methods, etc., a means for automatically linking with each service and production software, a means for generating revenue by realizing optimal self-coordination, a means for using an emotion engine that recognizes the user's emotions, and a means for purchasing or reserving suggested items on the spot. This makes it possible to suggest optimal self-coordination that takes the user's emotions into consideration, and to immediately purchase or reserve suggested items.

[0314] "Means for capturing user images and requirements" refers to means for obtaining information from the user using a camera, microphone, input interface, etc.

[0315] "Prompt design methods" are methods that use acquired information to suggest youthfulness, optimal products and purchasing methods, beauty and hairstyles, health maintenance methods, clothing, training methods, etc.

[0316] "Means for realizing automatic integration with each service and production software" refers to a means for automatically purchasing products or reserving services based on the proposed content.

[0317] "A means of generating revenue by realizing optimal self-coordination" is a means of generating revenue by creating specific coordination based on the proposed content and presenting it to users.

[0318] "Means using an emotion engine to recognize user emotions" refers to means using an engine to recognize emotions from the user's facial expressions and tone of voice.

[0319] "Means for instantly purchasing or reserving the proposed item" means means for instantly purchasing or reserving the proposed item.

[0320] The system for implementing this invention includes means for capturing a user's image and requirements, prompt design means, means for realizing automatic linkage with each service and production software, means for realizing optimal self-coordination, means for using an emotion engine that recognizes the user's emotions, and means for purchasing or reserving suggested items on the spot.

[0321] System program processing explanation

[0322] Hardware and Software

[0323] Hardware: Smartphone built-in camera and microphone

[0324] software:

[0325] OpenCV: Image capture and processing

[0326] EmotionRecognizer: Emotion Recognition Model

[0327] FashionRecommender: A model for suggesting fashion coordination

[0328] ShoppingAPI: Product purchase and reservation API

[0329] Processing flow

[0330] 1. A way to capture user images and requirements:

[0331] It uses the smartphone's camera and microphone to capture the user's image and voice, thereby obtaining information about the user's appearance and requirements from their voice.

[0332] 2. Prompt design methods:

[0333] Based on the captured image and voice information of the user, a generative AI model is used to generate prompts that suggest youthfulness, optimal products and purchasing methods, beauty and hairstyles, health maintenance methods, clothing, and training methods.

[0334] 3. Using the Emotion Engine:

[0335] The captured image and audio are input into EmotionRecognizer to recognize the user's emotions. For example, if the user is smiling, it is recognized as "joy," if they are frowning, it is recognized as "anger," and if the tone of their voice is low, it is recognized as "sadness."

[0336] 4. Means for realizing automatic integration with various services and production software:

[0337] Based on the proposals generated by the prompt design tool, the system works in conjunction with each service and production software to automatically purchase products and reserve services.

[0338] 5. Ways to realize optimal self-coordination:

[0339] FashionRecommender is used to suggest optimal fashion coordination based on the user's emotions and requirements. The suggested coordination is visually presented to the user.

[0340] 6. How to purchase / reserve the proposed items on the spot:

[0341] The suggested items can be purchased or reserved on the spot using the Shopping API, allowing users to get the suggested items instantly.

[0342] Examples of specific examples and prompts

[0343] For example, here is a prompt for a user who prefers a casual style and wants blue items:

[0344] Example prompt sentence:

[0345] Capture a user's image and recognize their emotions. They prefer casual style and want blue items. Suggest the perfect outfit and purchase / reserve the suggested items on the spot.

[0346] In this way, it becomes possible to suggest optimal self-coordination ideas that take into account the user's emotions, and to instantly purchase or reserve the suggested items.

[0347] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0348] Step 1:

[0349] A user captures their own image and voice using the smartphone's camera and microphone.

[0350] Input: User image and voice

[0351] Output: Captured image and audio data

[0352] Specific actions: The user launches the camera app on their smartphone, points their face at the camera to take a picture, and speaks their requirements into the microphone.

[0353] Step 2:

[0354] The server receives the captured image and audio data and analyzes the images using image processing software (OpenCV).

[0355] Input: Captured image and audio data

[0356] Output: Analyzed image and audio information

[0357] Specific operation: The server inputs image data into OpenCV to perform facial recognition and feature extraction, and converts voice data into text.

[0358] Step 3:

[0359] The server uses EmotionRecognizer to recognize the user's emotions from the analyzed image information.

[0360] Input: Parsed image information

[0361] Output: Recognized emotion data

[0362] Specific operation: The server inputs image information into EmotionRecognizer, which estimates the user's emotion from their facial expression. For example, a smile indicates "joy," while a furrowed brow indicates "anger."

[0363] Step 4:

[0364] The server uses the generative AI model to generate prompts that provide optimal suggestions based on the user's emotions and requirements.

[0365] Input: Recognized emotion data and audio information (textual requirements)

[0366] Output: Generated prompt statement

[0367] Specific operation: The server inputs the emotion data and requirements into the generative AI model to generate a prompt sentence, such as "The user prefers a casual style and would like blue items. Please suggest the best fashion coordination."

[0368] Step 5:

[0369] The server uses FashionRecommender to suggest the best fashion coordination based on the generated prompt sentence.

[0370] Input: Generated prompt statement

[0371] Output: Proposed fashion outfit

[0372] Specific operation: The server inputs the prompt sentence into the FashionRecommender, which selects fashion items based on the user's requirements and emotions, such as a casual blue shirt and jeans.

[0373] Step 6:

[0374] The server presents the proposed fashion coordination to the user and uses the Shopping API to process the purchase or reservation on the spot.

[0375] Input: Proposed fashion outfit

[0376] Output: Purchased / reserved items

[0377] Specific operation: The server displays the suggested items on the user's smartphone, and when the user selects to purchase or reserve, the purchase / reservation procedure is completed through the Shopping API.

[0378] Example 2

[0379] Next, a description will be given of Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0380] In conventional systems, even if users input their own images and requirements, appropriate suggestions, automatic purchasing procedures, and suggestions based on emotions were not made, resulting in low user satisfaction and the inability to achieve efficient self-coordination. In addition, since suggestions were not made taking into account the user's emotions, the system was unable to fully meet the user's needs.

[0381] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0382] In this invention, the server includes a means for capturing a user's image and requirements, a prompt design means for suggesting youthfulness, optimal products and purchasing methods, beauty and hairstyles, health maintenance methods, clothing, training methods, etc., a means for realizing automatic linkage with each service and production software, a means for generating revenue by realizing optimal self-coordination, a means for using an emotion engine that recognizes the user's emotions, and a means for making suggestions based on emotions. This enables appropriate suggestions based on the user's input information, automatic purchasing procedures, and suggestions based on emotions, thereby increasing user satisfaction and enabling efficient self-coordination.

[0383] The "means for capturing user images and requirements" is an interface through which a user can input their own image and specific requirements into the system and transmit that information to the server.

[0384] The "prompt design tool" is a function that suggests youthfulness, optimal products and purchasing methods, beauty and hairstyles, health maintenance methods, clothing, training methods, etc. based on the information entered by the user.

[0385] "Automatic linking means" is a function that automatically links with each service or production software to purchase products or reserve services based on the user's selection.

[0386] "Means for realizing optimal self-coordination" is a function for creating specific coordination based on selected products and services and the user's image, and presenting it to the user.

[0387] The "emotion engine" is an analytical engine that recognizes the user's emotions and makes suggestions based on those emotions.

[0388] "Means for making suggestions based on emotions" is a function that suggests optimal products and services, beauty and hairstyles, health maintenance methods, clothing, training methods, etc. based on the user's emotions recognized by the emotion engine.

[0389] This invention is a system that allows users to input their own image and requirements, and then suggests optimal products and services, beauty and hairstyles, health maintenance methods, clothing, training methods, etc. for maintaining a youthful appearance, and also automatically handles purchase procedures and service reservations. Specific embodiments of this system are described below.

[0390] System configuration

[0391] The system consists of the following main components:

[0392] 1. A way to capture user images and requirements

[0393] 2. Prompt Design Methods

[0394] 3. Automatic linking methods

[0395] 4. Means of realizing optimal self-coordination

[0396] 5. Emotion Engine

[0397] 6. Emotion-based suggestions

[0398] Hardware and Software

[0399] Terminal: The device through which the user enters images and requirements (e.g. smartphone, PC).

[0400] Server: A central processing unit for data processing and proposal generation.

[0401] Generative AI model: An artificial intelligence model for generating prompts based on user requirements.

[0402] Emotion Engine: An analysis engine for recognizing user emotions.

[0403] Image Processing Software: Software that superimposes selected products and services onto a user's image.

[0404] Data processing and calculation

[0405] 1. Obtaining user input information

[0406] The user uses the device to take a photo of themselves and enters the requirement that they want to look "youthful."

[0407] The terminal sends this information to the server.

[0408] 2. Proposal by prompt design method

[0409] The server uses a generative AI model to generate prompts based on the user's input.

[0410] Based on the generated prompts, it suggests youthful clothing, hairstyles, and ways to stay healthy.

[0411] 3. User Choice

[0412] The user uses the device to select the outfit they like from the suggestions.

[0413] The device sends the user's selection to the server.

[0414] 4. Purchasing procedures using automated linking methods

[0415] The server uses the online shop's API to complete the purchase of the selected outfit.

[0416] 5. Realizing optimal self-coordination

[0417] The server uses image processing software to combine the selected outfit with the user's image and sends the result to the device.

[0418] 6. Emotion Recognition by Emotion Engine

[0419] The server uses an emotion engine to recognize the user's emotion.

[0420] 7. Emotion-based recommendations

[0421] The server uses the generative AI model again to generate emotionally appropriate prompts based on the emotions recognized by the emotion engine.

[0422] Specific examples

[0423] A user takes a picture of themselves with their smartphone and enters "I want to look youthful." The device sends this information to a server, which uses a generative AI model to suggest youthful, casual clothing. The user selects a red jacket and the device sends the selection to the server. The server uses an online shop's API to complete the purchase of the red jacket and uses the Adobe Photoshop API to superimpose the red jacket onto the user's image. If the user expresses the emotion "joy," the server uses an emotion engine to recognize the emotion and uses a generative AI model to suggest bright-colored clothing and exercise methods that match the happy mood.

[0424] Prompt Sentence Examples

[0425] "The user inputs their own image and their desired youthful appearance. Please suggest youthful clothing, hairstyles, and health maintenance methods, automatically complete the procedure for purchasing the selected clothing from an online shop, and suggest the best self-coordination."

[0426] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0427] Step 1:

[0428] The user enters their image and requirements.

[0429] Input: User image, youthfulness requirement

[0430] Specific actions: The user takes a picture using a smartphone or PC and enters "I want to look younger" into the text box.

[0431] Output: The user's image and requirements are saved on the device and sent to the server.

[0432] Step 2:

[0433] The server generates the suggestions using a prompt design tool.

[0434] Input: User image, youthfulness requirement

[0435] What it does: The server uses the generative AI model to generate prompts based on the user's requirements.

[0436] Data processing: A generative AI model analyzes the user's image and requirements to generate suggestions for youthful clothing, hairstyles, and health maintenance methods.

[0437] Output: The suggestions are generated in a list format and sent to the device.

[0438] Step 3:

[0439] The user selects the outfit they like from the suggestions.

[0440] Input: A list of suggested outfits

[0441] What happens: The user clicks to select an item from the on-screen suggestion list.

[0442] Output: The user's selection is saved on the device and sent to the server.

[0443] Step 4:

[0444] The server performs the purchase procedure using an automatic linking means.

[0445] Input: User's chosen outfit

[0446] Specific operation: The server calls the online shop's API and sends the purchase information for the selected item.

[0447] Data processing: The server sends the information necessary for the purchase procedure to the online shop and completes the purchase.

[0448] Output: Purchase confirmation information is stored on the server and sent to the device.

[0449] Step 5:

[0450] The server embodies the optimal self-coordination.

[0451] Input: User image, selected outfit

[0452] What it does: The server uses image processing software to composite the selected outfit onto the user's image.

[0453] Data processing: Image processing software combines the user's image with the selected outfit to generate an outfit image.

[0454] Output: The generated coordinated image is sent to the device.

[0455] Step 6:

[0456] The server uses an emotion engine to recognize the user's emotions.

[0457] Input: User facial expressions and text input

[0458] Specific operation: The server uses an emotion engine to analyze emotions from the user's facial expressions and text.

[0459] Data processing: The emotion engine generates user emotion data.

[0460] Output: The recognized emotion data is stored on the server.

[0461] Step 7:

[0462] The server makes suggestions based on emotions.

[0463] Input: Recognized emotion data

[0464] Specific operation: The server uses the generative AI model again based on the emotion data to generate a prompt appropriate to the emotion.

[0465] Data processing: A generative AI model analyzes the emotion data and generates emotion-based recommendations.

[0466] Output: Sentiment-based suggestions are sent to the device.

[0467] Step 8:

[0468] The server uses an automatic linking method to make purchases and reservations.

[0469] Input: Suggestions based on user choices and sentiment data

[0470] Specific operation: The server calls the API of the online shop or service provider and sends purchase or reservation information.

[0471] Data processing: The server sends the information necessary for the purchase or reservation process to the online shop or service provider to complete the process.

[0472] Output: Purchase and reservation confirmation information is stored on the server and sent to the device.

[0473] Step 9:

[0474] The server embodies the optimal self-coordination.

[0475] Input: User image, selected products and services, emotional data

[0476] What it does: The server uses image processing software to composite the selected products and services onto the user's image.

[0477] Data processing: Image processing software combines the user's image with the selected products and services to generate a coordinated image.

[0478] Output: The generated coordinated image is sent to the device.

[0479] (Application example 2)

[0480] Next, a description will be given of Application Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0481] Conventional online shopping systems have had the problem of being difficult to recommend optimal products and complete purchasing procedures based on the user's individual needs and emotions. Furthermore, users cannot receive specific outfit suggestions based on their own images or emotions, making it difficult to stimulate purchase motivation. Furthermore, there is a lack of a way to automatically purchase the products selected by the user, making the system less convenient.

[0482] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0483] In this invention, the server includes means for capturing a user's image and requirements, prompt design means for suggesting youthfulness, optimal products and purchasing methods, beauty and hairstyles, health maintenance methods, clothing, training methods, etc., means for automatically linking with each service and production software, means for generating revenue by realizing optimal self-coordination, means for analyzing the user's emotions using an emotion recognition engine, means for generating suggestions based on the user's image features and emotions using a generative AI model, and means for automatically connecting to an online shop selling the suggested products and carrying out the purchase procedure. This enables optimal product suggestions and purchase procedures based on the user's individual requests and emotions.

[0484] "Means for capturing user images and requirements" refers to an interface through which a user can input their own image and specific requests into the system.

[0485] "Prompt design tools" are functions that suggest youthful clothing, hairstyles, and health maintenance methods based on the user's images, requirements, and emotions.

[0486] "Means for realizing automatic linkage" refers to a function that automatically links with external systems or online shops that offer suggested products or services, and allows for purchase procedures and reservations.

[0487] "Means for realizing optimal self-coordination" is a function for generating specific coordination based on selected products and services and the user's images and emotions, and presenting it to the user.

[0488] An "emotion recognition engine" is a technology that analyzes emotions from a user's images and voice, and recognizes emotional states such as "joy" and "sadness."

[0489] A "generative AI model" is an artificial intelligence technology that generates optimal suggestions based on the user's image features and emotions.

[0490] "Means for automatically connecting to an online store and completing the purchase process" is a function that automatically connects to an online store that sells the product selected by the user and automates the purchase process.

[0491] The system for implementing this invention includes means for capturing a user's image and requirements, prompt design means, means for realizing automatic coordination, means for realizing optimal self-coordination, an emotion recognition engine, a generative AI model, and means for automatically connecting to an online shop and carrying out the purchase process.

[0492] System configuration

[0493] 1. A way to capture user images and requirements

[0494] Users use a smartphone app to upload an image of themselves and enter their desired youthful appearance.

[0495] Use an image recognition library (e.g., OpenCV) to analyze the user's facial and clothing features.

[0496] 2. Prompt Design Methods

[0497] Based on the user's image characteristics and requirements, the app suggests youthful clothing, hairstyles, and health maintenance methods.

[0498] Use a generative AI model (e.g., OpenAI GPT-4) to generate suggestions.

[0499] Example: "Based on the user's image and emotions, suggest youthful clothing, hairstyles, and ways to stay healthy."

[0500] 3. Emotion Recognition Engine

[0501] Analyze user emotions using an emotion recognition engine (e.g. Microsoft Azure Emotion API).

[0502] Based on the analysis results, the user's emotions are classified as "joy" or "sadness," etc.

[0503] 4. Generative AI Models

[0504] Using a generative AI model, suggestions are generated based on the user's image features and emotions.

[0505] Example: "Based on the user's image and emotions, suggest youthful clothing, hairstyles, and ways to stay healthy."

[0506] 5. Means of realizing automatic collaboration

[0507] It automatically connects with external systems and online shops that offer suggested products and services to carry out purchase procedures and reservations.

[0508] Use online store APIs to automate the checkout process.

[0509] 6. Means of realizing optimal self-coordination

[0510] Based on the selected products and services, the user's images, and their emotions, a specific coordination is generated and presented to the user.

[0511] Specific examples

[0512] The user uploads a photo of themselves to a smartphone app and enters the phrase "I want to look youthful." The emotion recognition engine recognizes the emotion of "joy," and the generative AI model generates suggestions for "light-colored casual clothing and jogging that matches the joyful mood." The user selects the suggested outfit, and the app automatically connects to the online shop to complete the purchase.

[0513] In this way, optimal product suggestions and purchasing procedures can be made based on the user's individual needs and emotions.

[0514] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0515] Step 1:

[0516] Users upload their own images to a smartphone app and input their desired youthful appearance.

[0517] Input: User image, youthfulness requirement

[0518] Output: Image data, requirements data

[0519] What it does: The user takes a picture of themselves using the smartphone's camera and enters their requirements into the app's input form, which then sends the data to the server.

[0520] Step 2:

[0521] The server uses an image recognition library (e.g., OpenCV) to analyze the user's facial and clothing features.

[0522] Input: Image data

[0523] Output: Face feature data, clothing feature data

[0524] Specific operation: The server analyzes the received image data and extracts features such as facial contours, clothing color, and shape. These feature data are used for subsequent processing.

[0525] Step 3:

[0526] The server uses an emotion recognition engine (e.g., Microsoft Azure Emotion API) to analyze the user's emotions.

[0527] Input: Facial feature data

[0528] Output: Emotion data (e.g., happiness, sadness)

[0529] Specific operation: The server inputs facial feature data into an emotion recognition engine to analyze the user's emotional state. As a result of the analysis, the user's emotion is classified as "happiness" or "sadness," etc.

[0530] Step 4:

[0531] The server uses a generative AI model (e.g., OpenAI GPT-4) to generate suggestions based on the user's image features and emotions.

[0532] Input: Facial feature data, clothing feature data, emotion data, requirement data

[0533] Output: Suggested data (e.g., youthful clothing, hairstyles, and health maintenance methods)

[0534] Specific operation: The server inputs the user's image features, emotions, and requirements into the generative AI model to generate youthful suggestions, including clothing, hairstyles, and health maintenance methods suitable for the user.

[0535] Step 5:

[0536] The server automatically connects with external systems and online shops that offer suggested products and services, and handles purchase procedures and reservations.

[0537] Input: Proposed data, user selected data

[0538] Output: Purchase procedure data, reservation data

[0539] Specific operation: When the user selects a suggested product or service, the server automatically uses the online shop API to purchase the selected product and, if necessary, reserve the service.

[0540] Step 6:

[0541] The server generates specific coordination based on the selected products and services and the user's images and emotions, and presents it to the user.

[0542] Input: Selection data, facial feature data, clothing feature data, emotion data

[0543] Output: Coordinate data

[0544] Specific operation: The server generates a specific outfit based on the selected products and services, the user's image features, and their emotions. The generated outfit is visually presented to the user, allowing them to achieve optimal self-coordination.

[0545] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0546] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (registered trademark) (Internet search engine).<URL: https: / / openai.com / blog / chatgpt> ) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0547] Another example of generative AI is Gemini (registered trademark) (Internet search engine). <url: https: gemini.google.com ?hl="ja">) are listed.

[0548] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0549] [Second embodiment]

[0550] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0551] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0552] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0553] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0554] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0555] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0556] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0557] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0558] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0559] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0560] In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0561] Next, the specific processing by the specific processing unit 290 of the data processing device 12 will be described.

[0562] "Example 1"

[0563] The system of the present invention includes a means for capturing a user's image and requirements, a prompt design means, a means for automatically linking with various services and production software, and a means for realizing optimal self-coordination. The means for capturing a user's image and requirements acquires information from the user using a camera, microphone, input interface, etc. The prompt design means suggests youthfulness, optimal products and purchasing methods, beauty and hairstyles, health maintenance methods, clothing, training methods, etc. based on the acquired information. The means for automatically linking with various services and production software automatically purchases products and reserves services based on the suggested content. The means for realizing optimal self-coordination creates specific coordination based on the suggested content and presents it to the user.

[0564] "Example 2"

[0565] As a specific example, a user inputs an image of themselves and their desire to look youthful into the system. Based on this information, the prompt design means suggests youthful clothing, hairstyles, and health maintenance methods. When the user selects an outfit they like from the suggestions, the automatic linking means automatically connects to an online shop selling the selected outfit and completes the purchase process. Furthermore, the means for realizing optimal self-coordination creates a specific outfit based on the selected outfit and the user's image and presents it to the user.

[0566] The processing flow of each embodiment will be described below.

[0567] "Example 1"

[0568] Step 1: The user inputs their own image and requirements (e.g., a youthful appearance) into the system. This input is done using a camera, microphone, input interface, etc.

[0569] Step 2: The prompt design tool uses the input information to suggest youthful clothing, hairstyles, and health maintenance methods. These suggestions are based on the user's age, gender, body type, preferences, and other information.

[0570] Step 3: The user selects the outfit they like from the suggestions.

[0571] Step 4: The automatic linking means automatically connects to the online shop selling the selected clothing and carries out the purchase procedure.

[0572] Step 5: The means for realizing the optimal self-coordination creates a specific coordination based on the selected outfit and the user's image and presents it to the user.

[0573] "Example 2"

[0574] Step 1: The user inputs their image and requirements (e.g., health maintenance) into the system. This input is done using a camera, microphone, input interface, etc.

[0575] Step 2: The prompt design tool uses the input information to suggest healthy diets and exercise methods. These suggestions are based on the user's age, gender, body type, preferences, and other information.

[0576] Step 3: The user selects their preferred diet and exercise plan from the suggestions.

[0577] Step 4: An automatic linking means automatically makes reservations at restaurants that serve the selected meals and at training gyms that are suitable for the selected exercise method.

[0578] Step 5: The means for realizing optimal self-coordination creates a specific lifestyle based on the selected diet and exercise method and the user's image, and presents it to the user.

[0579] Example 1

[0580] Next, a description will be given of Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0581] Conventional self-coordination systems struggled to provide suggestions based on the individual requirements and characteristics of users, and lacked automatic coordination to translate the suggestions into concrete actions. As a result, users had to perform many manual operations, resulting in low convenience. Furthermore, the suggestions were not sufficiently detailed or visually presented, and there was a lack of ingenuity to make them easy for users to understand. This led to issues such as lower user satisfaction and a decrease in the frequency of system use.

[0582] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0583] In this invention, the server includes a means for capturing a user's image and requirements, a prompt design means for suggesting youthfulness, optimal products and purchasing methods, beauty and hairstyles, health maintenance methods, clothing, and training methods, a means for automatically linking with various services and production software, a means for generating revenue by realizing optimal self-coordination plans, a means for analyzing acquired image data using facial recognition technology, a means for converting voice data into text using speech recognition technology, a means for generating prompts using a generative AI model, a means for creating specific self-coordination plans based on the suggested content, and a means for presenting the generated self-coordination plans to the user. This enables suggestions based on the user's individual requirements and characteristics, realizes automatic linkage to translate the suggested content into concrete actions, and improves user convenience. Furthermore, the concreteness and visual presentation of the suggested content improves user understanding, improving user satisfaction and increasing system usage frequency.

[0584] "Means for capturing user images and requirements" refers to devices and software for obtaining image data, audio data, and text data from users using a camera, microphone, input interface, etc.

[0585] "Prompt design tools" are algorithms and software that use acquired user information to suggest youthfulness, optimal products and purchasing methods, beauty and hairstyles, health maintenance methods, clothing, training methods, etc.

[0586] "Means for realizing automatic integration with each service and production software" refers to an interface or API that automatically connects with external services such as online shops and reservation systems based on the proposed content, allowing products to be purchased or services to be reserved.

[0587] "Means for generating revenue by realizing optimal self-coordination" refers to a method or system for generating revenue by creating specific self-coordination based on proposed content and presenting it to users, thereby encouraging them to purchase products or use services.

[0588] "Means of analysis using facial recognition technology" refers to algorithms or software for extracting a user's age, gender, facial features, etc. from the acquired image data.

[0589] "Means of converting voice data into text using speech recognition technology" refers to algorithms or software that convert acquired voice data into text data and understand the user's requirements.

[0590] "Means for generating prompts using a generative AI model" refers to an artificial intelligence model or software that generates prompts containing suggestions based on the analysis results in accordance with the user's requirements.

[0591] The "means for creating specific self-coordination" refers to image processing technology and software that synthesizes clothing and hairstyles with the user's image based on the proposed content, generating a finished image.

[0592] The "means for presenting the generated self-coordinated image to the user" refers to an interface or software for transmitting the generated self-coordinated image to the user's terminal and displaying it.

[0593] This invention is a system that includes a means for capturing a user's image and requirements, a prompt design means, a means for automatically linking with various services and production software, and a means for realizing optimal self-coordination. Specific embodiments of this system are described below.

[0594] Retrieving User Information

[0595] Users access the system using a device (such as a smartphone or PC). They take a photo of themselves using the device's camera and input their requirements by voice using the microphone. Requirements can also be input as text using a keyboard or touch screen. For example, a user can take a photo of themselves using the smartphone's camera and say, "I'd like suggestions for a youthful style" using the microphone.

[0596] Sending data

[0597] The device sends the acquired image data, audio data, and text data to the server via an internet connection, such as Wi-Fi or mobile data.

[0598] Data analysis

[0599] The server analyzes the received data. For image data, it uses facial recognition technology (e.g., OpenCV) to extract the user's age, gender, and facial features. For voice data, it uses speech recognition technology (e.g., Google Speech-to-Text API) to convert it into text and understand the user's requirements. For text data, it uses natural language processing technology (e.g., spaCy) to analyze the requirements.

[0600] Generate prompt statement

[0601] The server generates a prompt using a generative AI model (e.g., GPT-4) based on the analysis results. For example, if a user inputs "Please suggest a youthful style," the server generates the prompt "Please suggest a youthful casual style for women in their 30s."

[0602] Service collaboration

[0603] The server then connects to various services and production software based on the generated prompt text. For example, it uses an online shop's API to search for suggested outfits and proceed with the purchase. It also connects with a hair salon's reservation system to make a reservation for a hairstyle.

[0604] Coordination details

[0605] The server then creates a specific self-coordinated outfit based on the suggestions. For example, it uses image processing software to combine the suggested clothing and hairstyle with the user's photo to create a finished image.

[0606] Presentation of results

[0607] The server sends the generated self-coordinated image to the terminal, which displays the results to the user. The user can then review the proposed coordinated outfit and make purchases or reservations as necessary.

[0608] In this way, users can easily find the perfect outfit for themselves. A concrete example of a prompt sentence is, "Please suggest a youthful, casual style for a woman in her 30s." This system makes suggestions based on the user's individual requirements and characteristics, and automatically links the suggestions to concrete actions. This improves user convenience and enhances user satisfaction by making the suggestions more concrete and visually presenting them.

[0609] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0610] Step 1:

[0611] Users access the system using a terminal. They take their own photo with the terminal's camera and input their requirements by voice using the microphone. They can also input requirements in text using a keyboard or touch screen. Input data can be image data, voice data, or text data.

[0612] Step 2:

[0613] The device transmits the acquired image data, voice data, and text data to the server via an Internet connection. The input is the data acquired from the user, and the output is the data sent to the server.

[0614] Step 3:

[0615] The server analyzes the received image data using facial recognition technology (e.g., OpenCV) to extract the user's age, gender, and facial features. The input is image data, and the output is the user's feature data as the analysis result.

[0616] Step 4:

[0617] The server converts the received voice data into text using speech recognition technology (e.g., Google Speech-to-Text API) to understand the user's requirements. The input is voice data, and the output is text data.

[0618] Step 5:

[0619] The server analyzes the received text data using natural language processing technology (e.g., spaCy) to understand the user's requirements. The input is text data, and the output is requirement data as the analysis result.

[0620] Step 6:

[0621] The server generates a prompt sentence using a generative AI model (e.g., GPT-4) based on the analysis results. For example, if a user inputs "Please suggest a youthful style," the server generates the prompt sentence "Please suggest a youthful casual style for women in their 30s." The input is the analysis result, and the output is the prompt sentence.

[0622] Step 7:

[0623] The server interacts with various services and production software based on the generated prompt text. For example, it uses an online shop's API to search for suggested outfits and proceed with the purchase process. It also interacts with a hair salon's reservation system to make a hairstyle reservation. The input is the prompt text, and the output is the result of the service interaction.

[0624] Step 8:

[0625] The server creates a specific self-coordinated outfit based on the proposed content. For example, it uses image processing software to combine the suggested clothing and hairstyle with the user's photo to generate a finished image. The input is the proposed content and the user's photo, and the output is the finished image.

[0626] Step 9:

[0627] The server sends the generated self-coordinated image to the terminal, which displays the result to the user. The input is the completed image, and the output is the display to the user. The user can review the proposed coordinated image and make purchases or reservations as necessary.

[0628] (Application example 1)

[0629] Next, a description will be given of Application Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0630] Conventional self-coordination systems have the problem that it is difficult for users to find the suggested products in physical stores, resulting in an inefficient shopping experience. Additionally, while there are systems that suggest optimal outfits based on the user's images and requirements, they lack the functionality to provide navigation information within physical stores, making it difficult for users to quickly find the suggested products.

[0631] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means. In this invention, the server includes a means for capturing a user's image and requirements, a prompt design means for suggesting youthfulness, optimal products and purchasing methods, beauty and hairstyles, health maintenance methods, clothing, and training methods, etc., a means for automatically linking with each service and production software, a means for generating revenue by realizing optimal self-coordination, and a means for providing navigation information for finding suggested products in a physical store. This allows the user to quickly find the suggested products in the physical store, making the shopping experience efficient and enjoyable.

[0632] "Means for capturing user images and requirements" refers to means for obtaining information from the user using a camera, microphone, input interface, etc.

[0633] "Prompt design methods" are methods that use acquired information to suggest youthfulness, optimal products and purchasing methods, beauty and hairstyles, health maintenance methods, clothing, training methods, etc.

[0634] "Means for realizing automatic integration with each service and production software" refers to a means for automatically purchasing products or reserving services based on the proposed content.

[0635] "A means of generating revenue by realizing optimal self-coordination" is a means of generating revenue by creating specific coordination based on the proposed content and presenting it to users.

[0636] The "means for providing navigation information for finding suggested products in a physical store" refers to a means for navigating so that suggested products can be easily found in a physical store.

[0637] As an embodiment of the present invention, the following system is constructed.

[0638] First, the smartphone's camera, microphone, and input interface are used to capture the user's image and requirements. The user takes a picture of themselves using the smartphone's camera and imports it into the app, which then inputs the user's image and requirements into the system.

[0639] Next, as a prompt design method, based on the acquired user image and requirements, the system suggests youthfulness, optimal products and purchasing methods, beauty and hairstyles, health maintenance methods, clothing, training methods, etc. For this, a generative AI model using Keras and TensorFlow is used. Specifically, the user's image is input into the AI ​​model, and a prompt is generated that suggests the optimal outfit.

[0640] Furthermore, as a means of achieving automatic integration with various services and production software, products can be purchased or services can be reserved automatically based on the proposed content, allowing users to easily purchase the proposed products or reserve the services.

[0641] In addition, as a means of generating revenue by realizing optimal self-coordination, specific coordinations are created based on the proposed content and presented to users, allowing them to actually try on the suggested coordinations.

[0642] Finally, as a means for providing navigation information for finding the suggested product in a physical store, navigation is provided so that the suggested product can be easily found in the physical store. Specifically, based on the suggested product, information is provided for navigating to where the product is located in the physical store.

[0643] For example, a user takes a picture of themselves with their smartphone camera and imports it into the app. The app then suggests, "The recommended outfit is a shirt, pants, and shoes." In a physical store, the app navigates to, "The shirt is in Aisle 3, the pants are in Aisle 5, and the shoes are in Aisle 7."

[0644] Examples of prompts include, "Based on the user's image, suggest the best fashion outfit" and "Based on the suggested outfit, provide navigation information to find the product in a physical store."

[0645] In this way, users can quickly find suggested products in physical stores, making their shopping experience efficient and enjoyable.

[0646] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0647] Step 1:

[0648] The user takes a picture of themselves with the smartphone camera and imports it into the app. The input is the user's image, and the output is image data. Specifically, the smartphone's camera app is launched and the user takes a picture of themselves. The captured image is saved in the app.

[0649] Step 2:

[0650] The image data captured by the device is input into a generative AI model to generate a prompt. The input is image data, and the output is a prompt. Specifically, the image data is input into a generative AI model using Keras and TensorFlow, and the model analyzes the image and generates a prompt that suggests the optimal outfit.

[0651] Step 3:

[0652] Based on the generated prompt, the server suggests the most suitable products and purchasing methods for the user, beauty and hairstyles, health maintenance methods, clothing, training methods, etc. The input is the prompt, and the output is the suggestions. Specific operations include analyzing the prompt and suggesting specific products and services to the user.

[0653] Step 4:

[0654] The server automatically connects with each service and production software based on the proposed content. The input is the proposed content, and the output is product purchase or service reservation information. Specifically, it calls an API to automatically purchase or reserve the proposed product or service.

[0655] Step 5:

[0656] The server creates the optimal self-coordinated outfit and presents it to the user. The input is the proposed content, and the output is specific coordinated information. Specifically, the server presents the specific coordinated outfit to the user in the form of images and text based on the proposed content.

[0657] Step 6:

[0658] The terminal provides navigation information to find the suggested product in the physical store. The input is the suggested product information, and the output is navigation information. Specifically, based on the suggested product, it provides information to navigate to the location of the product in the physical store. The user can navigate around the store and find the suggested product while looking at the smartphone screen.

[0659] Example 2

[0660] Next, a description will be given of Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0661] In conventional systems, even if users input their own images and requirements, it was difficult for them to receive appropriate suggestions, and the purchasing process for the suggested products and services was cumbersome. Furthermore, there was a lack of means for users to specifically check the clothes and hairstyles they selected, making it difficult to achieve optimal self-coordination.

[0662] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0663] In this invention, the server includes means for importing a user's image and requirements, a prompt design means for suggesting youthfulness, optimal products and purchasing methods, beauty and hairstyles, health maintenance methods, clothing, training methods, etc., a means for automatically linking with each service and production software, a means for generating monetization by realizing optimal self-coordination, a means for generating suggestions using a generative AI model, a means for connecting to an online shop based on the user's selection and completing the purchase process, a means for generating coordinated images based on the selected clothing and the user's image, and a means for presenting the generated coordinated images to the user. This allows the user to easily receive appropriate suggestions, smoothly complete the purchase process for the selected products and services, and further view specific coordinated images.

[0664] "Means for capturing user images and requirements" refers to a function for receiving image data and requirement information provided by the user and converting them into a format that can be processed within the system.

[0665] The "prompt design means" is a function that generates prompt sentences to be input into the generative AI model based on information provided by the user.

[0666] "Means for realizing automatic collaboration with each service and production software" refers to a function that enables the system to automatically collaborate with external services and software to exchange and process data.

[0667] "A means of generating revenue by realizing optimal self-coordination" is a function that shows specific coordinated outfits suggested to users and generates revenue as a result.

[0668] A "means for generating proposals using a generative AI model" is a function that uses a generative AI model to automatically generate proposals based on user requirements.

[0669] "Means of connecting to an online store based on the user's selection and completing the purchase process" is a function that automatically connects to an online store that sells the product selected by the user and completes the purchase process.

[0670] The "means for generating a coordinated image based on selected clothing and the user's image" is a function for combining the clothing selected by the user with the user's image to generate a specific coordinated image.

[0671] The "means for presenting the generated coordinated image to the user" is a function for displaying the generated coordinated image on the user's terminal so that the user can check it.

[0672] This invention is a system that allows users to input their own image and requirements, and then suggests youthful clothing, hairstyles, health maintenance methods, etc., and generates specific coordinated images. This system operates in cooperation with the server, terminal, and user.

[0673] First, the user uses a device such as a smartphone or PC to take or select a photo of themselves and input the requirement of "looking youthful." The device then sends the image and requirements entered by the user to the server. Specifically, the data is sent using an HTTP POST request.

[0674] The server generates a prompt sentence to be input to the generative AI model based on the received image and requirements. For example, it generates a prompt sentence such as "Please suggest youthful clothing, hairstyles, and health maintenance methods for this user." The server uses a generative AI model (e.g., OpenAI's GPT-4) to generate suggestions based on the prompt sentence. The suggestions include youthful clothing, hairstyles, and health maintenance methods.

[0675] The generated suggestions are sent from the server to the device. The user selects their favorite outfit and hairstyle from the suggestion list displayed on the device. For example, they select "white shirt and denim pants." The device then sends the user's selected data to the server. Specifically, it sends the data again using an HTTP POST request.

[0676] The server connects to the API of the online shop selling the selected outfit and automatically completes the purchase process. Specifically, it adds the item to the cart and proceeds with the purchase. The server then uses image processing software to generate a coordinated image based on the selected outfit and the user's image.

[0677] The generated outfit image is sent from the server to the device. The user can check the outfit image displayed on the device and visually confirm the specific outfit. For example, an image of the user wearing a white shirt and denim pants is displayed on the smartphone screen.

[0678] This system allows users to easily receive appropriate suggestions, smoothly complete the purchasing process for selected products and services, and even view specific outfit images.

[0679] As a concrete example, consider the case where a user takes a photo of themselves with their smartphone and inputs, "I'd like suggestions for youthful clothing." The server uses a generative AI model to suggest "white shirt and denim pants, short hairstyle, daily jogging." The user selects "white shirt and denim pants," and the server completes the purchase process using the online shop's API. The server combines the white shirt and denim pants with the user's image to generate an outfit image. An image of the user wearing the white shirt and denim pants is displayed on the user's smartphone.

[0680] Examples of prompts include, "Suggest youthful clothing, hairstyles, and health maintenance methods for this user" and "Create the perfect outfit based on the user's image."

[0681] In this way, users can simply enter their own image and requirements to receive suggestions for youthful clothing, hairstyles, and health maintenance methods, and can also view specific outfit images.

[0682] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0683] Step 1:

[0684] The user inputs the image and requirements.

[0685] Users use a smartphone or PC to take or select a photo of themselves and input their desired "youthfulness" criteria. The input data consists of an image file and text data.

[0686] Step 2:

[0687] The terminal sends the input data to the server.

[0688] The terminal sends the image file and text data entered by the user to the server via an HTTP POST request. The input is the user's image and requirements, and the output is data sent to the server.

[0689] Step 3:

[0690] The server generates a prompt.

[0691] The server analyzes the received image file and text data and generates a prompt such as, "Please suggest youthful clothing, hairstyles, and health maintenance methods for this user." The input is the image file and text data, and the output is the prompt.

[0692] Step 4:

[0693] The server generates suggestions using a generative AI model.

[0694] The server inputs a prompt into a generative AI model (e.g., OpenAI's GPT-4) and generates suggestions for youthful clothing, hairstyles, and health maintenance methods. The input is the prompt, and the output is the suggestion data.

[0695] Step 5:

[0696] The server sends the proposal to the device.

[0697] The server sends the generated proposal to the terminal in JSON format. The input is the proposal data, and the output is the data sent to the terminal.

[0698] Step 6:

[0699] The user selects from the suggestions.

[0700] The user selects the clothes and hairstyle they like from the suggestion list displayed on the terminal. For example, they select "white shirt and denim pants." The input is the suggestion list, and the output is the selected data.

[0701] Step 7:

[0702] The terminal transmits the selection data to the server.

[0703] The terminal sends the data of the user's selection, "white shirt and denim pants," to the server via an HTTP POST request. The input is the selected data, and the output is the data sent to the server.

[0704] Step 8:

[0705] The server connects to the online shop and completes the purchase.

[0706] The server connects to the API of the online shop selling the selected clothing and automatically completes the purchase process. Specifically, it adds the item to the cart and proceeds with the purchase. The input is the selected data, and the output is the completion of the purchase process.

[0707] Step 9:

[0708] The server generates a coordinated image.

[0709] The server generates a coordinated image using image processing software based on the selected outfit and the user's image. The input is the user's image and the selected outfit, and the output is the coordinated image.

[0710] Step 10:

[0711] The server transmits the coordinated image to the terminal.

[0712] The server sends the generated coordinated image to the terminal as an HTTP response. The input is the coordinated image, and the output is data sent to the terminal.

[0713] Step 11:

[0714] The user checks the coordinated outfit image.

[0715] The user checks the outfit image displayed on the device and visually confirms the specific outfit. For example, an image of the user wearing a white shirt and denim pants is displayed on the smartphone screen. The input is the outfit image, and the output is the user's confirmation.

[0716] (Application example 2)

[0717] Next, a description will be given of Application Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0718] In conventional online shopping and styling suggestion systems, even when users input their own images and requirements, the suggested products and styles often did not meet the user's expectations. Furthermore, the process of actually purchasing the suggested products was cumbersome, and there was a lack of mechanisms for users to easily make purchases. Furthermore, users were unable to try on the selected outfits, often resulting in dissatisfaction after purchase. To solve these problems, a system is needed that suggests optimal products and styles based on the user's images and requirements, facilitates the purchasing process, and provides a virtual try-on function.

[0719] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0720] In this invention, the server includes means for capturing a user's image and requirements, a prompt design means for suggesting youthfulness, optimal products and purchasing methods, beauty and hairstyles, health maintenance methods, clothing, training methods, etc., a means for automatically linking with each service and production software, a means for generating revenue by realizing optimal self-coordination, a means for creating specific coordination based on the user's image and selected clothing and presenting it to the user through a virtual try-on function, and a means for generating prompts using a generative AI model.This allows the user to receive suggestions for optimal products and styles based on their image and requirements, easily complete the purchase process, and try on the selected clothing through the virtual try-on function.

[0721] "Means for capturing user images and requirements" refers to an interface or device that allows a user to input their own image and desired requirements into the system.

[0722] "Prompt design tools" are algorithms and software that suggest youthfulness, optimal products, beauty and hairstyles, health maintenance methods, clothing, training methods, etc. based on the user's images and requirements.

[0723] "Automatic linking means" refers to a system that automatically connects the product or service selected by the user to the relevant online shop or service provider to complete the purchase procedure or make a reservation.

[0724] The "means for realizing self-coordination" refers to software or algorithms that create specific coordination based on the user's image and selected clothing and present it to the user.

[0725] The "virtual try-on feature" is a technology that allows users to see the outfit they have selected superimposed on their own image, making it appear as if they are trying it on.

[0726] A "generative AI model" is an artificial intelligence model that generates prompts based on the user's images and requirements and makes optimal suggestions.

[0727] The following system configuration is used as an embodiment of the present invention.

[0728] The server includes a means for capturing user images and requirements, a prompt design means, an automatic linking means, a means for realizing self-coordination, a virtual try-on function, and a generative AI model.

[0729] Users use devices such as smartphones or head-mounted displays to take a photo of themselves and input their desired youthful appearance, which is then sent to the server.

[0730] The server receives the user's image and requirements using a means for capturing the user's image and requirements, and then uses a prompt design means to suggest youthful clothing, hairstyles, health maintenance methods, etc. based on the user's image and requirements. The prompt design means uses a generative AI model to generate prompts and make optimal suggestions.

[0731] When a user selects an outfit they like from the suggestions, an automatic linking means is activated, automatically connecting them to the online shop selling the selected outfit and carrying out the purchase procedure, allowing the user to easily purchase the item.

[0732] Furthermore, a means for realizing self-coordination is used to create a specific outfit based on the user's image and the selected outfit. This outfit is presented to the user through a virtual try-on function. The user can see the selected outfit superimposed on their own image, allowing them to check it as if they were trying it on.

[0733] For example, if a user takes a photo of themselves with their smartphone and enters the requirement for "youthful clothing," the server will suggest a youthful casual jacket. When the user selects this jacket, the server automatically connects to the online shop and completes the purchase. The server then combines the user's image with an image of the selected jacket to create a specific outfit and presents it to the user through a virtual try-on function.

[0734] An example prompt is:

[0735] Enter the user image: path / to / user_image.jpg

[0736] Enter your youthfulness requirements: Youthful clothing

[0737] Choose the suggested outfit: Youthful casual jacket

[0738] This way, users can easily find, shop and try on youthful styles.

[0739] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0740] Step 1:

[0741] Users use a device such as a smartphone or head-mounted display to take a photo of themselves and input their desired youthful appearance.

[0742] Input: User image, youthfulness requirement

[0743] Output: User image data, requirement data

[0744] Specific behavior: Use the device's camera function to take a picture and enter the requirements in the text input field.

[0745] Step 2:

[0746] The terminal transmits the user's image data and requirement data to the server.

[0747] Input: User image data, requirement data

[0748] Output: Image data and requirements data sent to the server

[0749] Specific operation: Using the terminal's communication function, image data and requirement data are uploaded to the server.

[0750] Step 3:

[0751] The server receives the user's image data and requirement data, and proposes youthful clothing, hairstyles, health maintenance methods, etc. using prompt design means.

[0752] Input: User image data, requirement data

[0753] Output: A list of suggested youthful clothing, hairstyles, and health maintenance methods

[0754] What it does: It uses a generative AI model to generate prompts and make optimal suggestions based on the user's image and requirements.

[0755] Step 4:

[0756] The user selects the outfit they like from the suggestions.

[0757] Input: A list of suggested outfits

[0758] Output: Selected outfit

[0759] What it does: Use the device interface to select from a list of suggested outfits.

[0760] Step 5:

[0761] The server uses an automatic linking means to automatically connect to an online shop selling the selected clothing and carry out the purchase procedure.

[0762] Input: selected outfit

[0763] Output: Purchase completion notification

[0764] Specific operation: The server uses the online shop's API to automatically complete the purchase process for the selected clothing.

[0765] Step 6:

[0766] The server uses a means for realizing self-coordination to create a specific coordination based on the user's image and selected clothes.

[0767] Input: User image data, selected outfit

[0768] Output: Specific coordinated image

[0769] Specific operation: Using an image processing algorithm, the system combines the user's image with the image of the selected outfit to generate an outfit image.

[0770] Step 7:

[0771] The server presents specific outfit images to the user through a virtual try-on function.

[0772] Input: Specific coordinated image

[0773] Output: Coordinate image presented to the user

[0774] Specific operation: Coordinated images are displayed on the device display, allowing users to check the outfits as if they were trying them on.

[0775] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0776] "Example 1"

[0777] One embodiment of the present invention is a system that incorporates an emotion engine. This system uses a camera, microphone, input interface, etc. as a means of capturing a user's image and requirements. Furthermore, this capturing means is equipped with an emotion engine, which can recognize emotions from the user's facial expressions and tone of voice. For example, if the user is smiling, it is recognized as "joy," if they frown, it is "anger," and if their voice is low-pitched, it is "sadness."

[0778] "Example 2"

[0779] Next, based on the captured user information and the emotions recognized by the emotion engine, the prompt design tool suggests youthfulness, optimal products and purchasing methods, beauty and hairstyles, health maintenance methods, clothing, training methods, etc. For example, if the user expresses the emotion of "joy," it will suggest bright-colored clothing that the user would like and exercise methods that match their happy mood.

[0780] "Example 4"

[0781] Furthermore, the automatic linking means automatically purchases products or reserves services based on the user's selection and emotions. For example, if the user expresses "joy," the system automatically connects to an online shop selling brightly colored clothing selected by the user and completes the purchase. If the user expresses "sad," the system automatically reserves a relaxation service selected by the user.

[0782] "Example 5"

[0783] Finally, the means for realizing optimal self-coordination creates specific coordination based on the selected products and services, the user's image, and the emotions recognized by the emotion engine, and presents it to the user. For example, for a user who expresses the emotion of "joy," it will present an outfit that combines brightly colored clothing with an exercise method that matches the user's happy mood.

[0784] The processing flow of each embodiment will be described below.

[0785] "Example 1"

[0786] Step 1: The system captures the user's image and requirements using a camera, microphone, input interface, etc.

[0787] Step 2: The emotion engine built into the capture device recognizes emotions from the user's facial expressions and tone of voice.

[0788] "Example 2"

[0789] Step 1: The prompt design tool suggests youthfulness, optimal products and purchasing methods, beauty and hairstyles, health maintenance methods, clothing, training methods, etc. based on the captured user information and emotions recognized by the emotion engine.

[0790] Step 2: If the user expresses the emotion "joy," suggest bright-colored clothing that the user might like, or exercise methods that match their happy mood.

[0791] "Example 4"

[0792] Step 1: The automatic interaction means automatically purchases products or reserves services based on the user's choices and emotions.

[0793] Step 2: If the user shows the emotion "joy," the system automatically connects them to an online shop selling the brightly colored clothing they selected and completes the purchase. Step 3: If the user shows the emotion "sad," the system automatically makes a reservation for the relaxation service they selected.

[0794] "Example 5"

[0795] Step 1: The means for realizing the optimal self-coordination is to create a specific coordination based on the selected product or service, the user's image, and the emotions recognized by the emotion engine.

[0796] Step 2: For users who show the emotion of "joy," we suggest outfits that combine brightly colored clothing with exercise methods that match their happy mood.

[0797] Example 1

[0798] Next, a description will be given of Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0799] Conventional self-coordination systems struggled to provide proposals that fully reflected the individual needs and feelings of users, and lacked a means to visualize the proposed content in concrete terms. This made the process from the user understanding the proposals to actually taking action complicated, making it difficult to improve the user experience. Furthermore, linking with various services and production software was often done manually, making efficient operation difficult.

[0800] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0801] In this invention, the server includes a means for capturing a user's image and requirements, a prompt design means for suggesting youthfulness, optimal products and purchasing methods, beauty and hairstyles, health maintenance methods, clothing, and training methods, a means for automatically linking with various services and production software, a means for generating revenue by realizing optimal self-coordination, a means including an emotion engine for recognizing emotions from the user's facial expressions and tone of voice, a means for generating prompts using a generative AI model, and a means for generating virtual try-on images based on the suggested content. This enables suggestions that reflect the user's individual requests and emotions, and the concrete visualization of the suggested content improves the user experience. Furthermore, automatic linking with various services and production software enables efficient operation.

[0802] "Means for capturing user images and requirements" refers to means for obtaining information from the user using a camera, microphone, or input interface.

[0803] "Prompt design methods" are methods that use acquired information to suggest youthfulness, optimal products and purchasing methods, beauty and hairstyles, health maintenance methods, clothing, training methods, etc.

[0804] "Means for realizing automatic integration with each service and production software" refers to a means for automatically purchasing products or reserving services based on the proposed content.

[0805] The "means for realizing optimal self-coordination" is a means for creating a specific coordination based on the proposed content and presenting it to the user.

[0806] "Means including an emotion engine" refers to means that has the function of recognizing emotions from the user's facial expressions and tone of voice.

[0807] "Means for generating prompts using a generative AI model" refers to means for generating prompts using a generative AI model based on the analysis results.

[0808] The "means for generating a virtual try-on image" is a means for generating an image in which the user virtually tries on clothes based on the proposed content.

[0809] The system of the present invention includes a means for capturing a user's image and requirements, a prompt design means, a means for automatically linking with each service and production software, a means for realizing optimal self-coordination, a means including an emotion engine, a means for generating prompts using a generative AI model, and a means for generating virtual try-on images.

[0810] A way to capture user images and requirements

[0811] Users take a photo of themselves using the camera on their smartphone or computer, and then speak their request into the microphone. This data is temporarily stored on the device and then sent to the server.

[0812] Sending and storing information

[0813] The device sends the acquired image data and audio data to the server. The server stores the received data in a database. For example, image data is stored in the "user_images" table, and audio data is stored in the "user_requests" table.

[0814] Analysis of information

[0815] The server uses image analysis algorithms to estimate the user's age and gender. For example, it uses OpenCV for facial recognition and an age estimation model to estimate age. It also uses voice recognition software (e.g., Google Cloud Speech-to-Text) to convert voice data into text. It also uses an emotion engine (e.g., Microsoft Azure Emotion API) to recognize emotions from the user's facial expressions and tone of voice.

[0816] Prompt Generation

[0817] The server generates prompts using a generative AI model (e.g., GPT-4) based on the analysis results. For example, it generates a prompt such as, "For a smiling woman in her late 30s, suggest casual clothing that emphasizes youthfulness."

[0818] Service collaboration

[0819] The server then connects with various services based on the generated prompts. For example, it uses an online shop's API to search for suggested outfits and complete the purchase process. It also connects with a hair salon's reservation system to book the most suitable salon for the user.

[0820] Coordination details

[0821] The server then creates a specific self-coordinated outfit based on the suggested content, for example, by generating an image of the user virtually trying on the suggested outfit. For this purpose, it uses image processing software.

[0822] Presentation of results

[0823] The server then sends the generated virtual try-on images and the results of the purchase process to the device. The user can then check the outfit on their smartphone or computer screen. For example, the device will display a notification, and when the user opens the app, the virtual try-on images and a link to purchase will be displayed.

[0824] Examples of specific examples and prompts

[0825] As a concrete example, consider the case where a user takes a photo of themselves with their smartphone camera and then voice-inputs, "Please suggest some clothes that will make me look younger." The system estimates the user's age and gender, and uses an emotion engine to recognize that the user is smiling. Based on this, the system suggests casual clothes that emphasize youthfulness, automatically searches for the clothes in an online shop, and completes the purchase process. It also generates virtual try-on images and presents them to the user.

[0826] Example prompt sentence:

[0827] "Based on the user's image and voice input, please suggest casual clothing that emphasizes youthfulness. The user is smiling, in their late 30s, and female. Search for the suggested outfits in an online shop, complete the purchase process, and generate and present a virtual try-on image."

[0828] In this way, the system of the present invention collects and analyzes user information, generates prompts, and works with various services to provide optimal self-coordination.

[0829] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0830] Step 1:

[0831] Users take a photo of themselves using the camera on their smartphone or computer, and then input their request by voice using the microphone. The input data consists of image data and voice data, which are temporarily stored on the device.

[0832] Step 2:

[0833] The device sends the saved image data and audio data to the server. The server saves the received data in a database. For example, image data is saved in the "user_images" table, and audio data is saved in the "user_requests" table. The input is image data and audio data, and the output is saved in the database.

[0834] Step 3:

[0835] The server uses an image analysis algorithm to estimate the user's age and gender. For example, it uses OpenCV for facial recognition and estimates age using an age estimation model. The input is image data, and the output is the estimated age and gender. It also uses speech recognition software (e.g., Google Cloud Speech-to-Text) to convert voice data into text. The input is voice data, and the output is text data. It then uses an emotion engine (e.g., Microsoft Azure Emotion API) to recognize emotions from the user's facial expressions and tone of voice. The input is image data and voice data, and the output is the recognized emotion.

[0836] Step 4:

[0837] The server generates a prompt using a generative AI model (e.g., GPT-4) based on the analysis results. For example, it generates a prompt such as "Suggest casual clothing that emphasizes youthfulness to a smiling woman in her late 30s." The input is the estimated age, gender, emotion, and text data, and the output is the generated prompt.

[0838] Step 5:

[0839] The server interacts with various services based on the generated prompt. For example, it uses an online shop's API to search for suggested outfits and then completes the purchase process. The input is the generated prompt, and the output is the completion of the purchase process. It also interacts with a hair salon reservation system to make a reservation at the most suitable hair salon for the user. The input is the generated prompt, and the output is the completion of the reservation.

[0840] Step 6:

[0841] The server creates a specific self-coordinated outfit based on the suggested content. For example, it generates an image of the user virtually trying on the suggested outfit. For this purpose, it uses image processing software. The input is the user's image data and the suggested outfit data, and the output is a virtual try-on image.

[0842] Step 7:

[0843] The server sends the generated virtual try-on images and the results of the purchase process to the device. The user can then check the outfit on their smartphone or computer screen. The input is the virtual try-on images and the results of the purchase process, and the output is what is presented to the user. For example, the device displays a notification, and when the user opens the app, the virtual try-on images and a link to purchase are displayed.

[0844] (Application example 1)

[0845] Next, a description will be given of Application Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0846] Conventional self-coordination systems were unable to provide suggestions that took user emotions into consideration, and were therefore unable to fully increase user satisfaction. Furthermore, they lacked the ability to purchase or reserve suggested items on the spot, resulting in low user convenience. This limited the user's shopping experience and missed monetization opportunities.

[0847] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0848] In this invention, the server includes a means for capturing a user's image and requirements, a prompt design means for suggesting youthfulness, optimal products and purchasing methods, beauty and hairstyles, health maintenance methods, clothing, training methods, etc., a means for automatically linking with each service and production software, a means for generating revenue by realizing optimal self-coordination, a means for using an emotion engine that recognizes the user's emotions, and a means for purchasing or reserving suggested items on the spot. This makes it possible to suggest optimal self-coordination that takes the user's emotions into consideration, and to immediately purchase or reserve suggested items.

[0849] "Means for capturing user images and requirements" refers to means for obtaining information from the user using a camera, microphone, input interface, etc.

[0850] "Prompt design methods" are methods that use acquired information to suggest youthfulness, optimal products and purchasing methods, beauty and hairstyles, health maintenance methods, clothing, training methods, etc.

[0851] "Means for realizing automatic integration with each service and production software" refers to a means for automatically purchasing products or reserving services based on the proposed content.

[0852] "A means of generating revenue by realizing optimal self-coordination" is a means of generating revenue by creating specific coordination based on the proposed content and presenting it to users.

[0853] "Means using an emotion engine to recognize user emotions" refers to means using an engine to recognize emotions from the user's facial expressions and tone of voice.

[0854] "Means for instantly purchasing or reserving the proposed item" means means for instantly purchasing or reserving the proposed item.

[0855] The system for implementing this invention includes means for capturing a user's image and requirements, prompt design means, means for realizing automatic linkage with each service and production software, means for realizing optimal self-coordination, means for using an emotion engine that recognizes the user's emotions, and means for purchasing or reserving suggested items on the spot.

[0856] System program processing explanation

[0857] Hardware and Software

[0858] Hardware: Smartphone built-in camera and microphone

[0859] software:

[0860] OpenCV: Image capture and processing

[0861] EmotionRecognizer: Emotion Recognition Model

[0862] FashionRecommender: A model for suggesting fashion coordination

[0863] ShoppingAPI: Product purchase and reservation API

[0864] Processing flow

[0865] 1. A way to capture user images and requirements:

[0866] It uses the smartphone's camera and microphone to capture the user's image and voice, thereby obtaining information about the user's appearance and requirements from their voice.

[0867] 2. Prompt design methods:

[0868] Based on the captured image and voice information of the user, a generative AI model is used to generate prompts that suggest youthfulness, optimal products and purchasing methods, beauty and hairstyles, health maintenance methods, clothing, and training methods.

[0869] 3. Using the Emotion Engine:

[0870] The captured image and audio are input into EmotionRecognizer to recognize the user's emotions. For example, if the user is smiling, it is recognized as "joy," if they are frowning, it is recognized as "anger," and if the tone of their voice is low, it is recognized as "sadness."

[0871] 4. Means for realizing automatic integration with various services and production software:

[0872] Based on the proposals generated by the prompt design tool, the system works in conjunction with each service and production software to automatically purchase products and reserve services.

[0873] 5. Ways to realize optimal self-coordination:

[0874] FashionRecommender is used to suggest optimal fashion coordination based on the user's emotions and requirements. The suggested coordination is visually presented to the user.

[0875] 6. How to purchase / reserve the proposed items on the spot:

[0876] The suggested items can be purchased or reserved on the spot using the Shopping API, allowing users to get the suggested items instantly.

[0877] Examples of specific examples and prompts

[0878] For example, here is a prompt for a user who prefers a casual style and wants blue items:

[0879] Example prompt sentence:

[0880] Capture a user's image and recognize their emotions. They prefer casual style and want blue items. Suggest the perfect outfit and purchase / reserve the suggested items on the spot.

[0881] In this way, it becomes possible to suggest optimal self-coordination ideas that take into account the user's emotions, and to instantly purchase or reserve the suggested items.

[0882] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0883] Step 1:

[0884] A user captures their own image and voice using the smartphone's camera and microphone.

[0885] Input: User image and voice

[0886] Output: Captured image and audio data

[0887] Specific actions: The user launches the camera app on their smartphone, points their face at the camera to take a picture, and speaks their requirements into the microphone.

[0888] Step 2:

[0889] The server receives the captured image and audio data and analyzes the images using image processing software (OpenCV).

[0890] Input: Captured image and audio data

[0891] Output: Analyzed image and audio information

[0892] Specific operation: The server inputs image data into OpenCV to perform facial recognition and feature extraction, and converts voice data into text.

[0893] Step 3:

[0894] The server uses EmotionRecognizer to recognize the user's emotions from the analyzed image information.

[0895] Input: Parsed image information

[0896] Output: Recognized emotion data

[0897] Specific operation: The server inputs image information into EmotionRecognizer, which estimates the user's emotion from their facial expression. For example, a smile indicates "joy," while a furrowed brow indicates "anger."

[0898] Step 4:

[0899] The server uses the generative AI model to generate prompts that provide optimal suggestions based on the user's emotions and requirements.

[0900] Input: Recognized emotion data and audio information (textual requirements)

[0901] Output: Generated prompt statement

[0902] Specific operation: The server inputs the emotion data and requirements into the generative AI model to generate a prompt sentence, such as "The user prefers a casual style and would like blue items. Please suggest the best fashion coordination."

[0903] Step 5:

[0904] The server uses FashionRecommender to suggest the best fashion coordination based on the generated prompt sentence.

[0905] Input: Generated prompt statement

[0906] Output: Proposed fashion outfit

[0907] Specific operation: The server inputs the prompt sentence into the FashionRecommender, which selects fashion items based on the user's requirements and emotions, such as a casual blue shirt and jeans.

[0908] Step 6:

[0909] The server presents the proposed fashion coordination to the user and uses the Shopping API to process the purchase or reservation on the spot.

[0910] Input: Proposed fashion outfit

[0911] Output: Purchased / reserved items

[0912] Specific operation: The server displays the suggested items on the user's smartphone, and when the user selects to purchase or reserve, the purchase / reservation procedure is completed through the Shopping API.

[0913] Example 2

[0914] Next, a description will be given of Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0915] In conventional systems, even if users input their own images and requirements, appropriate suggestions, automatic purchasing procedures, and suggestions based on emotions were not made, resulting in low user satisfaction and the inability to achieve efficient self-coordination. In addition, since suggestions were not made taking into account the user's emotions, the system was unable to fully meet the user's needs.

[0916] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0917] In this invention, the server includes a means for capturing a user's image and requirements, a prompt design means for suggesting youthfulness, optimal products and purchasing methods, beauty and hairstyles, health maintenance methods, clothing, training methods, etc., a means for realizing automatic linkage with each service and production software, a means for generating revenue by realizing optimal self-coordination, a means for using an emotion engine that recognizes the user's emotions, and a means for making suggestions based on emotions. This enables appropriate suggestions based on the user's input information, automatic purchasing procedures, and suggestions based on emotions, thereby increasing user satisfaction and enabling efficient self-coordination.

[0918] The "means for capturing user images and requirements" is an interface through which a user can input their own image and specific requirements into the system and transmit that information to the server.

[0919] The "prompt design tool" is a function that suggests youthfulness, optimal products and purchasing methods, beauty and hairstyles, health maintenance methods, clothing, training methods, etc. based on the information entered by the user.

[0920] "Automatic linking means" is a function that automatically links with each service or production software to purchase products or reserve services based on the user's selection.

[0921] "Means for realizing optimal self-coordination" is a function for creating specific coordination based on selected products and services and the user's image, and presenting it to the user.

[0922] The "emotion engine" is an analytical engine that recognizes the user's emotions and makes suggestions based on those emotions.

[0923] "Means for making suggestions based on emotions" is a function that suggests optimal products and services, beauty and hairstyles, health maintenance methods, clothing, training methods, etc. based on the user's emotions recognized by the emotion engine.

[0924] This invention is a system that allows users to input their own image and requirements, and then suggests optimal products and services, beauty and hairstyles, health maintenance methods, clothing, training methods, etc. for maintaining a youthful appearance, and also automatically handles purchase procedures and service reservations. Specific embodiments of this system are described below.

[0925] System configuration

[0926] The system consists of the following main components:

[0927] 1. A way to capture user images and requirements

[0928] 2. Prompt Design Methods

[0929] 3. Automatic linking methods

[0930] 4. Means of realizing optimal self-coordination

[0931] 5. Emotion Engine

[0932] 6. Emotion-based suggestions

[0933] Hardware and Software

[0934] Terminal: The device through which the user enters images and requirements (e.g. smartphone, PC).

[0935] Server: A central processing unit for data processing and proposal generation.

[0936] Generative AI model: An artificial intelligence model for generating prompts based on user requirements.

[0937] Emotion Engine: An analysis engine for recognizing user emotions.

[0938] Image Processing Software: Software that superimposes selected products and services onto a user's image.

[0939] Data processing and calculation

[0940] 1. Obtaining user input information

[0941] The user uses the device to take a photo of themselves and enters the requirement that they want to look "youthful."

[0942] The terminal sends this information to the server.

[0943] 2. Proposal by prompt design method

[0944] The server uses a generative AI model to generate prompts based on the user's input.

[0945] Based on the generated prompts, it suggests youthful clothing, hairstyles, and ways to stay healthy.

[0946] 3. User Choice

[0947] The user uses the device to select the outfit they like from the suggestions.

[0948] The device sends the user's selection to the server.

[0949] 4. Purchasing procedures using automated linking methods

[0950] The server uses the online shop's API to complete the purchase of the selected outfit.

[0951] 5. Realizing optimal self-coordination

[0952] The server uses image processing software to combine the selected outfit with the user's image and sends the result to the device.

[0953] 6. Emotion Recognition by Emotion Engine

[0954] The server uses an emotion engine to recognize the user's emotion.

[0955] 7. Emotion-based recommendations

[0956] The server uses the generative AI model again to generate emotionally appropriate prompts based on the emotions recognized by the emotion engine.

[0957] Specific examples

[0958] A user takes a picture of themselves with their smartphone and enters "I want to look youthful." The device sends this information to a server, which uses a generative AI model to suggest youthful, casual clothing. The user selects a red jacket and the device sends the selection to the server. The server uses an online shop's API to complete the purchase of the red jacket and uses the Adobe Photoshop API to superimpose the red jacket onto the user's image. If the user expresses the emotion "joy," the server uses an emotion engine to recognize the emotion and uses a generative AI model to suggest bright-colored clothing and exercise methods that match the happy mood.

[0959] Prompt Sentence Examples

[0960] "The user inputs their own image and their desired youthful appearance. Please suggest youthful clothing, hairstyles, and health maintenance methods, automatically complete the procedure for purchasing the selected clothing from an online shop, and suggest the best self-coordination."

[0961] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0962] Step 1:

[0963] The user enters their image and requirements.

[0964] Input: User image, youthfulness requirement

[0965] Specific actions: The user takes a picture using a smartphone or PC and enters "I want to look younger" into the text box.

[0966] Output: The user's image and requirements are saved on the device and sent to the server.

[0967] Step 2:

[0968] The server generates the suggestions using a prompt design tool.

[0969] Input: User image, youthfulness requirement

[0970] What it does: The server uses the generative AI model to generate prompts based on the user's requirements.

[0971] Data processing: A generative AI model analyzes the user's image and requirements to generate suggestions for youthful clothing, hairstyles, and health maintenance methods.

[0972] Output: The suggestions are generated in a list format and sent to the device.

[0973] Step 3:

[0974] The user selects the outfit they like from the suggestions.

[0975] Input: A list of suggested outfits

[0976] What happens: The user clicks to select an item from the on-screen suggestion list.

[0977] Output: The user's selection is saved on the device and sent to the server.

[0978] Step 4:

[0979] The server performs the purchase procedure using an automatic linking means.

[0980] Input: User's chosen outfit

[0981] Specific operation: The server calls the online shop's API and sends the purchase information for the selected item.

[0982] Data processing: The server sends the information necessary for the purchase procedure to the online shop and completes the purchase.

[0983] Output: Purchase confirmation information is stored on the server and sent to the device.

[0984] Step 5:

[0985] The server embodies the optimal self-coordination.

[0986] Input: User image, selected outfit

[0987] What it does: The server uses image processing software to composite the selected outfit onto the user's image.

[0988] Data processing: Image processing software combines the user's image with the selected outfit to generate an outfit image.

[0989] Output: The generated coordinated image is sent to the device.

[0990] Step 6:

[0991] The server uses an emotion engine to recognize the user's emotions.

[0992] Input: User facial expressions and text input

[0993] Specific operation: The server uses an emotion engine to analyze emotions from the user's facial expressions and text.

[0994] Data processing: The emotion engine generates user emotion data.

[0995] Output: The recognized emotion data is stored on the server.

[0996] Step 7:

[0997] The server makes suggestions based on emotions.

[0998] Input: Recognized emotion data

[0999] Specific operation: The server uses the generative AI model again based on the emotion data to generate a prompt appropriate to the emotion.

[1000] Data processing: A generative AI model analyzes the emotion data and generates emotion-based recommendations.

[1001] Output: Sentiment-based suggestions are sent to the device.

[1002] Step 8:

[1003] The server uses an automatic linking method to make purchases and reservations.

[1004] Input: Suggestions based on user choices and sentiment data

[1005] Specific operation: The server calls the API of the online shop or service provider and sends purchase or reservation information.

[1006] Data processing: The server sends the information necessary for the purchase or reservation process to the online shop or service provider to complete the process.

[1007] Output: Purchase and reservation confirmation information is stored on the server and sent to the device.

[1008] Step 9:

[1009] The server embodies the optimal self-coordination.

[1010] Input: User image, selected products and services, emotional data

[1011] What it does: The server uses image processing software to composite the selected products and services onto the user's image.

[1012] Data processing: Image processing software combines the user's image with the selected products and services to generate a coordinated image.

[1013] Output: The generated coordinated image is sent to the device.

[1014] (Application example 2)

[1015] Next, a description will be given of Application Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1016] Conventional online shopping systems have had the problem of being difficult to recommend optimal products and complete purchasing procedures based on the user's individual needs and emotions. Furthermore, users cannot receive specific outfit suggestions based on their own images or emotions, making it difficult to stimulate purchase motivation. Furthermore, there is a lack of a way to automatically purchase the products selected by the user, making the system less convenient.

[1017] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1018] In this invention, the server includes means for capturing a user's image and requirements, prompt design means for suggesting youthfulness, optimal products and purchasing methods, beauty and hairstyles, health maintenance methods, clothing, training methods, etc., means for automatically linking with each service and production software, means for generating revenue by realizing optimal self-coordination, means for analyzing the user's emotions using an emotion recognition engine, means for generating suggestions based on the user's image features and emotions using a generative AI model, and means for automatically connecting to an online shop selling the suggested products and carrying out the purchase procedure. This enables optimal product suggestions and purchase procedures based on the user's individual requests and emotions.

[1019] "Means for capturing user images and requirements" refers to an interface through which a user can input their own image and specific requests into the system.

[1020] "Prompt design tools" are functions that suggest youthful clothing, hairstyles, and health maintenance methods based on the user's images, requirements, and emotions.

[1021] "Means for realizing automatic linkage" refers to a function that automatically links with external systems or online shops that offer suggested products or services, and allows for purchase procedures and reservations.

[1022] "Means for realizing optimal self-coordination" is a function for generating specific coordination based on selected products and services and the user's images and emotions, and presenting it to the user.

[1023] An "emotion recognition engine" is a technology that analyzes emotions from a user's images and voice, and recognizes emotional states such as "joy" and "sadness."

[1024] A "generative AI model" is an artificial intelligence technology that generates optimal suggestions based on the user's image features and emotions.

[1025] "Means for automatically connecting to an online store and completing the purchase process" is a function that automatically connects to an online store that sells the product selected by the user and automates the purchase process.

[1026] The system for implementing this invention includes means for capturing a user's image and requirements, prompt design means, means for realizing automatic coordination, means for realizing optimal self-coordination, an emotion recognition engine, a generative AI model, and means for automatically connecting to an online shop and carrying out the purchase process.

[1027] System configuration

[1028] 1. A way to capture user images and requirements

[1029] Users use a smartphone app to upload an image of themselves and enter their desired youthful appearance.

[1030] Use an image recognition library (e.g., OpenCV) to analyze the user's facial and clothing features.

[1031] 2. Prompt Design Methods

[1032] Based on the user's image characteristics and requirements, the app suggests youthful clothing, hairstyles, and health maintenance methods.

[1033] Use a generative AI model (e.g., OpenAI GPT-4) to generate suggestions.

[1034] Example: "Based on the user's image and emotions, suggest youthful clothing, hairstyles, and ways to stay healthy."

[1035] 3. Emotion Recognition Engine

[1036] Analyze user emotions using an emotion recognition engine (e.g. Microsoft Azure Emotion API).

[1037] Based on the analysis results, the user's emotions are classified as "joy" or "sadness," etc.

[1038] 4. Generative AI Models

[1039] Using a generative AI model, suggestions are generated based on the user's image features and emotions.

[1040] Example: "Based on the user's image and emotions, suggest youthful clothing, hairstyles, and ways to stay healthy."

[1041] 5. Means of realizing automatic collaboration

[1042] It automatically connects with external systems and online shops that offer suggested products and services to carry out purchase procedures and reservations.

[1043] Use online store APIs to automate the checkout process.

[1044] 6. Means of realizing optimal self-coordination

[1045] Based on the selected products and services, the user's images, and their emotions, a specific coordination is generated and presented to the user.

[1046] Specific examples

[1047] The user uploads a photo of themselves to a smartphone app and enters the phrase "I want to look youthful." The emotion recognition engine recognizes the emotion of "joy," and the generative AI model generates suggestions for "light-colored casual clothing and jogging that matches the joyful mood." The user selects the suggested outfit, and the app automatically connects to the online shop to complete the purchase.

[1048] In this way, optimal product suggestions and purchasing procedures can be made based on the user's individual needs and emotions.

[1049] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1050] Step 1:

[1051] Users upload their own images to a smartphone app and input their desired youthful appearance.

[1052] Input: User image, youthfulness requirement

[1053] Output: Image data, requirements data

[1054] What it does: The user takes a picture of themselves using the smartphone's camera and enters their requirements into the app's input form, which then sends the data to the server.

[1055] Step 2:

[1056] The server uses an image recognition library (e.g., OpenCV) to analyze the user's facial and clothing features.

[1057] Input: Image data

[1058] Output: Face feature data, clothing feature data

[1059] Specific operation: The server analyzes the received image data and extracts features such as facial contours, clothing color, and shape. These feature data are used for subsequent processing.

[1060] Step 3:

[1061] The server uses an emotion recognition engine (e.g., Microsoft Azure Emotion API) to analyze the user's emotions.

[1062] Input: Facial feature data

[1063] Output: Emotion data (e.g., happiness, sadness)

[1064] Specific operation: The server inputs facial feature data into an emotion recognition engine to analyze the user's emotional state. As a result of the analysis, the user's emotion is classified as "happiness" or "sadness," etc.

[1065] Step 4:

[1066] The server uses a generative AI model (e.g., OpenAI GPT-4) to generate suggestions based on the user's image features and emotions.

[1067] Input: Facial feature data, clothing feature data, emotion data, requirement data

[1068] Output: Suggested data (e.g., youthful clothing, hairstyles, and health maintenance methods)

[1069] Specific operation: The server inputs the user's image features, emotions, and requirements into the generative AI model to generate youthful suggestions, including clothing, hairstyles, and health maintenance methods suitable for the user.

[1070] Step 5:

[1071] The server automatically connects with external systems and online shops that offer suggested products and services, and handles purchase procedures and reservations.

[1072] Input: Proposed data, user selected data

[1073] Output: Purchase procedure data, reservation data

[1074] Specific operation: When the user selects a suggested product or service, the server automatically uses the online shop API to purchase the selected product and, if necessary, reserve the service.

[1075] Step 6:

[1076] The server generates specific coordination based on the selected products and services and the user's images and emotions, and presents it to the user.

[1077] Input: Selection data, facial feature data, clothing feature data, emotion data

[1078] Output: Coordinate data

[1079] Specific operation: The server generates a specific outfit based on the selected products and services, the user's image features, and their emotions. The generated outfit is visually presented to the user, allowing them to achieve optimal self-coordination.

[1080] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1081] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1082] Another example of generative AI is Gemini (internet search engine). <url: https: gemini.google.com ?hl="ja">) are listed.

[1083] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[1084] [Third embodiment]

[1085] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[1086] 5, the data processing system 310 includes the data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[1087] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1088] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[1089] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1090] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1091] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1092] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1093] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1094] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1095] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1096] Next, the specific processing by the specific processing unit 290 of the data processing device 12 will be described.

[1097] "Example 1"

[1098] The system of the present invention includes a means for capturing a user's image and requirements, a prompt design means, a means for automatically linking with various services and production software, and a means for realizing optimal self-coordination. The means for capturing a user's image and requirements acquires information from the user using a camera, microphone, input interface, etc. The prompt design means suggests youthfulness, optimal products and purchasing methods, beauty and hairstyles, health maintenance methods, clothing, training methods, etc. based on the acquired information. The means for automatically linking with various services and production software automatically purchases products and reserves services based on the suggested content. The means for realizing optimal self-coordination creates specific coordination based on the suggested content and presents it to the user.

[1099] "Example 2"

[1100] As a specific example, a user inputs an image of themselves and their desire to look youthful into the system. Based on this information, the prompt design means suggests youthful clothing, hairstyles, and health maintenance methods. When the user selects an outfit they like from the suggestions, the automatic linking means automatically connects to an online shop selling the selected outfit and completes the purchase process. Furthermore, the means for realizing optimal self-coordination creates a specific outfit based on the selected outfit and the user's image and presents it to the user.

[1101] The processing flow of each embodiment will be described below.

[1102] "Example 1"

[1103] Step 1: The user inputs their own image and requirements (e.g., a youthful appearance) into the system. This input is done using a camera, microphone, input interface, etc.

[1104] Step 2: The prompt design tool uses the input information to suggest youthful clothing, hairstyles, and health maintenance methods. These suggestions are based on the user's age, gender, body type, preferences, and other information.

[1105] Step 3: The user selects the outfit they like from the suggestions.

[1106] Step 4: The automatic linking means automatically connects to the online shop selling the selected clothing and carries out the purchase procedure.

[1107] Step 5: The means for realizing the optimal self-coordination creates a specific coordination based on the selected outfit and the user's image and presents it to the user.

[1108] "Example 2"

[1109] Step 1: The user inputs their image and requirements (e.g., health maintenance) into the system. This input is done using a camera, microphone, input interface, etc.

[1110] Step 2: The prompt design tool uses the input information to suggest healthy diets and exercise methods. These suggestions are based on the user's age, gender, body type, preferences, and other information.

[1111] Step 3: The user selects their preferred diet and exercise plan from the suggestions.

[1112] Step 4: An automatic linking means automatically makes reservations at restaurants that serve the selected meals and at training gyms that are suitable for the selected exercise method.

[1113] Step 5: The means for realizing optimal self-coordination creates a specific lifestyle based on the selected diet and exercise method and the user's image, and presents it to the user.

[1114] Example 1

[1115] Next, a description will be given of Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1116] Conventional self-coordination systems struggled to provide suggestions based on the individual requirements and characteristics of users, and lacked automatic coordination to translate the suggestions into concrete actions. As a result, users had to perform many manual operations, resulting in low convenience. Furthermore, the suggestions were not sufficiently detailed or visually presented, and there was a lack of ingenuity to make them easy for users to understand. This led to issues such as lower user satisfaction and a decrease in the frequency of system use.

[1117] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1118] In this invention, the server includes a means for capturing a user's image and requirements, a prompt design means for suggesting youthfulness, optimal products and purchasing methods, beauty and hairstyles, health maintenance methods, clothing, and training methods, a means for automatically linking with various services and production software, a means for generating revenue by realizing optimal self-coordination plans, a means for analyzing acquired image data using facial recognition technology, a means for converting voice data into text using speech recognition technology, a means for generating prompts using a generative AI model, a means for creating specific self-coordination plans based on the suggested content, and a means for presenting the generated self-coordination plans to the user. This enables suggestions based on the user's individual requirements and characteristics, realizes automatic linkage to translate the suggested content into concrete actions, and improves user convenience. Furthermore, the concreteness and visual presentation of the suggested content improves user understanding, improving user satisfaction and increasing system usage frequency.

[1119] "Means for capturing user images and requirements" refers to devices and software for obtaining image data, audio data, and text data from users using a camera, microphone, input interface, etc.

[1120] "Prompt design tools" are algorithms and software that use acquired user information to suggest youthfulness, optimal products and purchasing methods, beauty and hairstyles, health maintenance methods, clothing, training methods, etc.

[1121] "Means for realizing automatic integration with each service and production software" refers to an interface or API that automatically connects with external services such as online shops and reservation systems based on the proposed content, allowing products to be purchased or services to be reserved.

[1122] "Means for generating revenue by realizing optimal self-coordination" refers to a method or system for generating revenue by creating specific self-coordination based on proposed content and presenting it to users, thereby encouraging them to purchase products or use services.

[1123] "Means of analysis using facial recognition technology" refers to algorithms or software for extracting a user's age, gender, facial features, etc. from the acquired image data.

[1124] "Means of converting voice data into text using speech recognition technology" refers to algorithms or software that convert acquired voice data into text data and understand the user's requirements.

[1125] "Means for generating prompts using a generative AI model" refers to an artificial intelligence model or software that generates prompts containing suggestions based on the analysis results in accordance with the user's requirements.

[1126] The "means for creating specific self-coordination" refers to image processing technology and software that synthesizes clothing and hairstyles with the user's image based on the proposed content, generating a finished image.

[1127] The "means for presenting the generated self-coordinated image to the user" refers to an interface or software for transmitting the generated self-coordinated image to the user's terminal and displaying it.

[1128] This invention is a system that includes a means for capturing a user's image and requirements, a prompt design means, a means for automatically linking with various services and production software, and a means for realizing optimal self-coordination. Specific embodiments of this system are described below.

[1129] Retrieving User Information

[1130] Users access the system using a device (such as a smartphone or PC). They take a photo of themselves using the device's camera and input their requirements by voice using the microphone. Requirements can also be input as text using a keyboard or touch screen. For example, a user can take a photo of themselves using the smartphone's camera and say, "I'd like suggestions for a youthful style" using the microphone.

[1131] Sending data

[1132] The device sends the acquired image data, audio data, and text data to the server via an internet connection, such as Wi-Fi or mobile data.

[1133] Data analysis

[1134] The server analyzes the received data. For image data, it uses facial recognition technology (e.g., OpenCV) to extract the user's age, gender, and facial features. For voice data, it uses speech recognition technology (e.g., Google Speech-to-Text API) to convert it into text and understand the user's requirements. For text data, it uses natural language processing technology (e.g., spaCy) to analyze the requirements.

[1135] Generate prompt statement

[1136] The server generates a prompt using a generative AI model (e.g., GPT-4) based on the analysis results. For example, if a user inputs "Please suggest a youthful style," the server generates the prompt "Please suggest a youthful casual style for women in their 30s."

[1137] Service collaboration

[1138] The server then connects to various services and production software based on the generated prompt text. For example, it uses an online shop's API to search for suggested outfits and proceed with the purchase. It also connects with a hair salon's reservation system to make a reservation for a hairstyle.

[1139] Coordination details

[1140] The server then creates a specific self-coordinated outfit based on the suggestions. For example, it uses image processing software to combine the suggested clothing and hairstyle with the user's photo to create a finished image.

[1141] Presentation of results

[1142] The server sends the generated self-coordinated image to the terminal, which displays the results to the user. The user can then review the proposed coordinated outfit and make purchases or reservations as necessary.

[1143] In this way, users can easily find the perfect outfit for themselves. A concrete example of a prompt sentence is, "Please suggest a youthful, casual style for a woman in her 30s." This system makes suggestions based on the user's individual requirements and characteristics, and automatically links the suggestions to concrete actions. This improves user convenience and enhances user satisfaction by making the suggestions more concrete and visually presenting them.

[1144] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1145] Step 1:

[1146] Users access the system using a terminal. They take their own photo with the terminal's camera and input their requirements by voice using the microphone. They can also input requirements in text using a keyboard or touch screen. Input data can be image data, voice data, or text data.

[1147] Step 2:

[1148] The device transmits the acquired image data, voice data, and text data to the server via an Internet connection. The input is the data acquired from the user, and the output is the data sent to the server.

[1149] Step 3:

[1150] The server analyzes the received image data using facial recognition technology (e.g., OpenCV) to extract the user's age, gender, and facial features. The input is image data, and the output is the user's feature data as the analysis result.

[1151] Step 4:

[1152] The server converts the received voice data into text using speech recognition technology (e.g., Google Speech-to-Text API) to understand the user's requirements. The input is voice data, and the output is text data.

[1153] Step 5:

[1154] The server analyzes the received text data using natural language processing technology (e.g., spaCy) to understand the user's requirements. The input is text data, and the output is requirement data as the analysis result.

[1155] Step 6:

[1156] The server generates a prompt sentence using a generative AI model (e.g., GPT-4) based on the analysis results. For example, if a user inputs "Please suggest a youthful style," the server generates the prompt sentence "Please suggest a youthful casual style for women in their 30s." The input is the analysis result, and the output is the prompt sentence.

[1157] Step 7:

[1158] The server interacts with various services and production software based on the generated prompt text. For example, it uses an online shop's API to search for suggested outfits and proceed with the purchase process. It also interacts with a hair salon's reservation system to make a hairstyle reservation. The input is the prompt text, and the output is the result of the service interaction.

[1159] Step 8:

[1160] The server creates a specific self-coordinated outfit based on the proposed content. For example, it uses image processing software to combine the suggested clothing and hairstyle with the user's photo to generate a finished image. The input is the proposed content and the user's photo, and the output is the finished image.

[1161] Step 9:

[1162] The server sends the generated self-coordinated image to the terminal, which displays the result to the user. The input is the completed image, and the output is the display to the user. The user can review the proposed coordinated image and make purchases or reservations as necessary.

[1163] (Application example 1)

[1164] Next, a description will be given of Application Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1165] Conventional self-coordination systems have the problem that it is difficult for users to find the suggested products in physical stores, resulting in an inefficient shopping experience. Additionally, while there are systems that suggest optimal outfits based on the user's images and requirements, they lack the functionality to provide navigation information within physical stores, making it difficult for users to quickly find the suggested products.

[1166] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means. In this invention, the server includes a means for capturing a user's image and requirements, a prompt design means for suggesting youthfulness, optimal products and purchasing methods, beauty and hairstyles, health maintenance methods, clothing, and training methods, etc., a means for automatically linking with each service and production software, a means for generating revenue by realizing optimal self-coordination, and a means for providing navigation information for finding suggested products in a physical store. This allows the user to quickly find the suggested products in the physical store, making the shopping experience efficient and enjoyable.

[1167] "Means for capturing user images and requirements" refers to means for obtaining information from the user using a camera, microphone, input interface, etc.

[1168] "Prompt design methods" are methods that use acquired information to suggest youthfulness, optimal products and purchasing methods, beauty and hairstyles, health maintenance methods, clothing, training methods, etc.

[1169] "Means for realizing automatic integration with each service and production software" refers to a means for automatically purchasing products or reserving services based on the proposed content.

[1170] "A means of generating revenue by realizing optimal self-coordination" is a means of generating revenue by creating specific coordination based on the proposed content and presenting it to users.

[1171] The "means for providing navigation information for finding suggested products in a physical store" refers to a means for navigating so that suggested products can be easily found in a physical store.

[1172] As an embodiment of the present invention, the following system is constructed.

[1173] First, the smartphone's camera, microphone, and input interface are used to capture the user's image and requirements. The user takes a picture of themselves using the smartphone's camera and imports it into the app, which then inputs the user's image and requirements into the system.

[1174] Next, as a prompt design method, based on the acquired user image and requirements, the system suggests youthfulness, optimal products and purchasing methods, beauty and hairstyles, health maintenance methods, clothing, training methods, etc. For this, a generative AI model using Keras and TensorFlow is used. Specifically, the user's image is input into the AI ​​model, and a prompt is generated that suggests the optimal outfit.

[1175] Furthermore, as a means of achieving automatic integration with various services and production software, products can be purchased or services can be reserved automatically based on the proposed content, allowing users to easily purchase the proposed products or reserve the services.

[1176] In addition, as a means of generating revenue by realizing optimal self-coordination, specific coordinations are created based on the proposed content and presented to users, allowing them to actually try on the suggested coordinations.

[1177] Finally, as a means for providing navigation information for finding the suggested product in a physical store, navigation is provided so that the suggested product can be easily found in the physical store. Specifically, based on the suggested product, information is provided for navigating to where the product is located in the physical store.

[1178] For example, a user takes a picture of themselves with their smartphone camera and imports it into the app. The app then suggests, "The recommended outfit is a shirt, pants, and shoes." In a physical store, the app navigates to, "The shirt is in Aisle 3, the pants are in Aisle 5, and the shoes are in Aisle 7."

[1179] Examples of prompts include, "Based on the user's image, suggest the best fashion outfit" and "Based on the suggested outfit, provide navigation information to find the product in a physical store."

[1180] In this way, users can quickly find suggested products in physical stores, making their shopping experience efficient and enjoyable.

[1181] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1182] Step 1:

[1183] The user takes a picture of themselves with the smartphone camera and imports it into the app. The input is the user's image, and the output is image data. Specifically, the smartphone's camera app is launched and the user takes a picture of themselves. The captured image is saved in the app.

[1184] Step 2:

[1185] The image data captured by the device is input into a generative AI model to generate a prompt. The input is image data, and the output is a prompt. Specifically, the image data is input into a generative AI model using Keras and TensorFlow, and the model analyzes the image and generates a prompt that suggests the optimal outfit.

[1186] Step 3:

[1187] Based on the generated prompt, the server suggests the most suitable products and purchasing methods for the user, beauty and hairstyles, health maintenance methods, clothing, training methods, etc. The input is the prompt, and the output is the suggestions. Specific operations include analyzing the prompt and suggesting specific products and services to the user.

[1188] Step 4:

[1189] The server automatically connects with each service and production software based on the proposed content. The input is the proposed content, and the output is product purchase or service reservation information. Specifically, it calls an API to automatically purchase or reserve the proposed product or service.

[1190] Step 5:

[1191] The server creates the optimal self-coordinated outfit and presents it to the user. The input is the proposed content, and the output is specific coordinated information. Specifically, the server presents the specific coordinated outfit to the user in the form of images and text based on the proposed content.

[1192] Step 6:

[1193] The terminal provides navigation information to find the suggested product in the physical store. The input is the suggested product information, and the output is navigation information. Specifically, based on the suggested product, it provides information to navigate to the location of the product in the physical store. The user can navigate around the store and find the suggested product while looking at the smartphone screen.

[1194] Example 2

[1195] Next, a description will be given of Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1196] In conventional systems, even if users input their own images and requirements, it was difficult for them to receive appropriate suggestions, and the purchasing process for the suggested products and services was cumbersome. Furthermore, there was a lack of means for users to specifically check the clothes and hairstyles they selected, making it difficult to achieve optimal self-coordination.

[1197] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1198] In this invention, the server includes means for importing a user's image and requirements, a prompt design means for suggesting youthfulness, optimal products and purchasing methods, beauty and hairstyles, health maintenance methods, clothing, training methods, etc., a means for automatically linking with each service and production software, a means for generating monetization by realizing optimal self-coordination, a means for generating suggestions using a generative AI model, a means for connecting to an online shop based on the user's selection and completing the purchase process, a means for generating coordinated images based on the selected clothing and the user's image, and a means for presenting the generated coordinated images to the user. This allows the user to easily receive appropriate suggestions, smoothly complete the purchase process for the selected products and services, and further view specific coordinated images.

[1199] "Means for capturing user images and requirements" refers to a function for receiving image data and requirement information provided by the user and converting them into a format that can be processed within the system.

[1200] The "prompt design means" is a function that generates prompt sentences to be input into the generative AI model based on information provided by the user.

[1201] "Means for realizing automatic collaboration with each service and production software" refers to a function that enables the system to automatically collaborate with external services and software to exchange and process data.

[1202] "A means of generating revenue by realizing optimal self-coordination" is a function that shows specific coordinated outfits suggested to users and generates revenue as a result.

[1203] A "means for generating proposals using a generative AI model" is a function that uses a generative AI model to automatically generate proposals based on user requirements.

[1204] "Means of connecting to an online store based on the user's selection and completing the purchase process" is a function that automatically connects to an online store that sells the product selected by the user and completes the purchase process.

[1205] The "means for generating a coordinated image based on selected clothing and the user's image" is a function for combining the clothing selected by the user with the user's image to generate a specific coordinated image.

[1206] The "means for presenting the generated coordinated image to the user" is a function for displaying the generated coordinated image on the user's terminal so that the user can check it.

[1207] This invention is a system that allows users to input their own image and requirements, and then suggests youthful clothing, hairstyles, health maintenance methods, etc., and generates specific coordinated images. This system operates in cooperation with the server, terminal, and user.

[1208] First, the user uses a device such as a smartphone or PC to take or select a photo of themselves and input the requirement of "looking youthful." The device then sends the image and requirements entered by the user to the server. Specifically, the data is sent using an HTTP POST request.

[1209] The server generates a prompt sentence to be input to the generative AI model based on the received image and requirements. For example, it generates a prompt sentence such as "Please suggest youthful clothing, hairstyles, and health maintenance methods for this user." The server uses a generative AI model (e.g., OpenAI's GPT-4) to generate suggestions based on the prompt sentence. The suggestions include youthful clothing, hairstyles, and health maintenance methods.

[1210] The generated suggestions are sent from the server to the device. The user selects their favorite outfit and hairstyle from the suggestion list displayed on the device. For example, they select "white shirt and denim pants." The device then sends the user's selected data to the server. Specifically, it sends the data again using an HTTP POST request.

[1211] The server connects to the API of the online shop selling the selected outfit and automatically completes the purchase process. Specifically, it adds the item to the cart and proceeds with the purchase. The server then uses image processing software to generate a coordinated image based on the selected outfit and the user's image.

[1212] The generated outfit image is sent from the server to the device. The user can check the outfit image displayed on the device and visually confirm the specific outfit. For example, an image of the user wearing a white shirt and denim pants is displayed on the smartphone screen.

[1213] This system allows users to easily receive appropriate suggestions, smoothly complete the purchasing process for selected products and services, and even view specific outfit images.

[1214] As a concrete example, consider the case where a user takes a photo of themselves with their smartphone and inputs, "I'd like suggestions for youthful clothing." The server uses a generative AI model to suggest "white shirt and denim pants, short hairstyle, daily jogging." The user selects "white shirt and denim pants," and the server completes the purchase process using the online shop's API. The server combines the white shirt and denim pants with the user's image to generate an outfit image. An image of the user wearing the white shirt and denim pants is displayed on the user's smartphone.

[1215] Examples of prompts include, "Suggest youthful clothing, hairstyles, and health maintenance methods for this user" and "Create the perfect outfit based on the user's image."

[1216] In this way, users can simply enter their own image and requirements to receive suggestions for youthful clothing, hairstyles, and health maintenance methods, and can also view specific outfit images.

[1217] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1218] Step 1:

[1219] The user inputs the image and requirements.

[1220] Users use a smartphone or PC to take or select a photo of themselves and input their desired "youthfulness" criteria. The input data consists of an image file and text data.

[1221] Step 2:

[1222] The terminal sends the input data to the server.

[1223] The terminal sends the image file and text data entered by the user to the server via an HTTP POST request. The input is the user's image and requirements, and the output is data sent to the server.

[1224] Step 3:

[1225] The server generates a prompt.

[1226] The server analyzes the received image file and text data and generates a prompt such as, "Please suggest youthful clothing, hairstyles, and health maintenance methods for this user." The input is the image file and text data, and the output is the prompt.

[1227] Step 4:

[1228] The server generates suggestions using a generative AI model.

[1229] The server inputs a prompt into a generative AI model (e.g., OpenAI's GPT-4) and generates suggestions for youthful clothing, hairstyles, and health maintenance methods. The input is the prompt, and the output is the suggestion data.

[1230] Step 5:

[1231] The server sends the proposal to the device.

[1232] The server sends the generated proposal to the terminal in JSON format. The input is the proposal data, and the output is the data sent to the terminal.

[1233] Step 6:

[1234] The user selects from the suggestions.

[1235] The user selects the clothes and hairstyle they like from the suggestion list displayed on the terminal. For example, they select "white shirt and denim pants." The input is the suggestion list, and the output is the selected data.

[1236] Step 7:

[1237] The terminal transmits the selection data to the server.

[1238] The terminal sends the data of the user's selection, "white shirt and denim pants," to the server via an HTTP POST request. The input is the selected data, and the output is the data sent to the server.

[1239] Step 8:

[1240] The server connects to the online shop and completes the purchase.

[1241] The server connects to the API of the online shop selling the selected clothing and automatically completes the purchase process. Specifically, it adds the item to the cart and proceeds with the purchase. The input is the selected data, and the output is the completion of the purchase process.

[1242] Step 9:

[1243] The server generates a coordinated image.

[1244] The server generates a coordinated image using image processing software based on the selected outfit and the user's image. The input is the user's image and the selected outfit, and the output is the coordinated image.

[1245] Step 10:

[1246] The server transmits the coordinated image to the terminal.

[1247] The server sends the generated coordinated image to the terminal as an HTTP response. The input is the coordinated image, and the output is data sent to the terminal.

[1248] Step 11:

[1249] The user checks the coordinated outfit image.

[1250] The user checks the outfit image displayed on the device and visually confirms the specific outfit. For example, an image of the user wearing a white shirt and denim pants is displayed on the smartphone screen. The input is the outfit image, and the output is the user's confirmation.

[1251] (Application example 2)

[1252] Next, a description will be given of Application Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1253] In conventional online shopping and styling suggestion systems, even when users input their own images and requirements, the suggested products and styles often did not meet the user's expectations. Furthermore, the process of actually purchasing the suggested products was cumbersome, and there was a lack of mechanisms for users to easily make purchases. Furthermore, users were unable to try on the selected outfits, often resulting in dissatisfaction after purchase. To solve these problems, a system is needed that suggests optimal products and styles based on the user's images and requirements, facilitates the purchasing process, and provides a virtual try-on function.

[1254] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1255] In this invention, the server includes means for capturing a user's image and requirements, a prompt design means for suggesting youthfulness, optimal products and purchasing methods, beauty and hairstyles, health maintenance methods, clothing, training methods, etc., a means for automatically linking with each service and production software, a means for generating revenue by realizing optimal self-coordination, a means for creating specific coordination based on the user's image and selected clothing and presenting it to the user through a virtual try-on function, and a means for generating prompts using a generative AI model.This allows the user to receive suggestions for optimal products and styles based on their image and requirements, easily complete the purchase process, and try on the selected clothing through the virtual try-on function.

[1256] "Means for capturing user images and requirements" refers to an interface or device that allows a user to input their own image and desired requirements into the system.

[1257] "Prompt design tools" are algorithms and software that suggest youthfulness, optimal products, beauty and hairstyles, health maintenance methods, clothing, training methods, etc. based on the user's images and requirements.

[1258] "Automatic linking means" refers to a system that automatically connects the product or service selected by the user to the relevant online shop or service provider to complete the purchase procedure or make a reservation.

[1259] The "means for realizing self-coordination" refers to software or algorithms that create specific coordination based on the user's image and selected clothing and present it to the user.

[1260] The "virtual try-on feature" is a technology that allows users to see the outfit they have selected superimposed on their own image, making it appear as if they are trying it on.

[1261] A "generative AI model" is an artificial intelligence model that generates prompts based on the user's images and requirements and makes optimal suggestions.

[1262] The following system configuration is used as an embodiment of the present invention.

[1263] The server includes a means for capturing user images and requirements, a prompt design means, an automatic linking means, a means for realizing self-coordination, a virtual try-on function, and a generative AI model.

[1264] Users use devices such as smartphones or head-mounted displays to take a photo of themselves and input their desired youthful appearance, which is then sent to the server.

[1265] The server receives the user's image and requirements using a means for capturing the user's image and requirements, and then uses a prompt design means to suggest youthful clothing, hairstyles, health maintenance methods, etc. based on the user's image and requirements. The prompt design means uses a generative AI model to generate prompts and make optimal suggestions.

[1266] When a user selects an outfit they like from the suggestions, an automatic linking means is activated, automatically connecting them to the online shop selling the selected outfit and carrying out the purchase procedure, allowing the user to easily purchase the item.

[1267] Furthermore, a means for realizing self-coordination is used to create a specific outfit based on the user's image and the selected outfit. This outfit is presented to the user through a virtual try-on function. The user can see the selected outfit superimposed on their own image, allowing them to check it as if they were trying it on.

[1268] For example, if a user takes a photo of themselves with their smartphone and enters the requirement for "youthful clothing," the server will suggest a youthful casual jacket. When the user selects this jacket, the server automatically connects to the online shop and completes the purchase. The server then combines the user's image with an image of the selected jacket to create a specific outfit and presents it to the user through a virtual try-on function.

[1269] An example prompt is:

[1270] Enter the user image: path / to / user_image.jpg

[1271] Enter your youthfulness requirements: Youthful clothing

[1272] Choose the suggested outfit: Youthful casual jacket

[1273] This way, users can easily find, shop and try on youthful styles.

[1274] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1275] Step 1:

[1276] Users use a device such as a smartphone or head-mounted display to take a photo of themselves and input their desired youthful appearance.

[1277] Input: User image, youthfulness requirement

[1278] Output: User image data, requirement data

[1279] Specific behavior: Use the device's camera function to take a picture and enter the requirements in the text input field.

[1280] Step 2:

[1281] The terminal transmits the user's image data and requirement data to the server.

[1282] Input: User image data, requirement data

[1283] Output: Image data and requirements data sent to the server

[1284] Specific operation: Using the terminal's communication function, image data and requirement data are uploaded to the server.

[1285] Step 3:

[1286] The server receives the user's image data and requirement data, and proposes youthful clothing, hairstyles, health maintenance methods, etc. using prompt design means.

[1287] Input: User image data, requirement data

[1288] Output: A list of suggested youthful clothing, hairstyles, and health maintenance methods

[1289] What it does: It uses a generative AI model to generate prompts and make optimal suggestions based on the user's image and requirements.

[1290] Step 4:

[1291] The user selects the outfit they like from the suggestions.

[1292] Input: A list of suggested outfits

[1293] Output: Selected outfit

[1294] What it does: Use the device interface to select from a list of suggested outfits.

[1295] Step 5:

[1296] The server uses an automatic linking means to automatically connect to an online shop selling the selected clothing and carry out the purchase procedure.

[1297] Input: selected outfit

[1298] Output: Purchase completion notification

[1299] Specific operation: The server uses the online shop's API to automatically complete the purchase process for the selected clothing.

[1300] Step 6:

[1301] The server uses a means for realizing self-coordination to create a specific coordination based on the user's image and selected clothes.

[1302] Input: User image data, selected outfit

[1303] Output: Specific coordinated image

[1304] Specific operation: Using an image processing algorithm, the system combines the user's image with the image of the selected outfit to generate an outfit image.

[1305] Step 7:

[1306] The server presents specific outfit images to the user through a virtual try-on function.

[1307] Input: Specific coordinated image

[1308] Output: Coordinate image presented to the user

[1309] Specific operation: Coordinated images are displayed on the device display, allowing users to check the outfits as if they were trying them on.

[1310] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1311] "Example 1"

[1312] One embodiment of the present invention is a system that incorporates an emotion engine. This system uses a camera, microphone, input interface, etc. as a means of capturing a user's image and requirements. Furthermore, this capturing means is equipped with an emotion engine, which can recognize emotions from the user's facial expressions and tone of voice. For example, if the user is smiling, it is recognized as "joy," if they frown, it is "anger," and if their voice is low-pitched, it is "sadness."

[1313] "Example 2"

[1314] Next, based on the captured user information and the emotions recognized by the emotion engine, the prompt design tool suggests youthfulness, optimal products and purchasing methods, beauty and hairstyles, health maintenance methods, clothing, training methods, etc. For example, if the user expresses the emotion of "joy," it will suggest bright-colored clothing that the user would like and exercise methods that match their happy mood.

[1315] "Example 4"

[1316] Furthermore, the automatic linking means automatically purchases products or reserves services based on the user's selection and emotions. For example, if the user expresses "joy," the system automatically connects to an online shop selling brightly colored clothing selected by the user and completes the purchase. If the user expresses "sad," the system automatically reserves a relaxation service selected by the user.

[1317] "Example 5"

[1318] Finally, the means for realizing optimal self-coordination creates specific coordination based on the selected products and services, the user's image, and the emotions recognized by the emotion engine, and presents it to the user. For example, for a user who expresses the emotion of "joy," it will present an outfit that combines brightly colored clothing with an exercise method that matches the user's happy mood.

[1319] The processing flow of each embodiment will be described below.

[1320] "Example 1"

[1321] Step 1: The system captures the user's image and requirements using a camera, microphone, input interface, etc.

[1322] Step 2: The emotion engine built into the capture device recognizes emotions from the user's facial expressions and tone of voice.

[1323] "Example 2"

[1324] Step 1: The prompt design tool suggests youthfulness, optimal products and purchasing methods, beauty and hairstyles, health maintenance methods, clothing, training methods, etc. based on the captured user information and emotions recognized by the emotion engine.

[1325] Step 2: If the user expresses the emotion "joy," suggest bright-colored clothing that the user might like, or exercise methods that match their happy mood.

[1326] "Example 4"

[1327] Step 1: The automatic interaction means automatically purchases products or reserves services based on the user's choices and emotions.

[1328] Step 2: If the user shows the emotion "joy," the system automatically connects them to an online shop selling the brightly colored clothing they selected and completes the purchase. Step 3: If the user shows the emotion "sad," the system automatically makes a reservation for the relaxation service they selected.

[1329] "Example 5"

[1330] Step 1: The means for realizing the optimal self-coordination is to create a specific coordination based on the selected product or service, the user's image, and the emotions recognized by the emotion engine.

[1331] Step 2: For users who show the emotion of "joy," we suggest outfits that combine brightly colored clothing with exercise methods that match their happy mood.

[1332] Example 1

[1333] Next, a description will be given of Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1334] Conventional self-coordination systems struggled to provide proposals that fully reflected the individual needs and feelings of users, and lacked a means to visualize the proposed content in concrete terms. This made the process from the user understanding the proposals to actually taking action complicated, making it difficult to improve the user experience. Furthermore, linking with various services and production software was often done manually, making efficient operation difficult.

[1335] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1336] In this invention, the server includes a means for capturing a user's image and requirements, a prompt design means for suggesting youthfulness, optimal products and purchasing methods, beauty and hairstyles, health maintenance methods, clothing, and training methods, a means for automatically linking with various services and production software, a means for generating revenue by realizing optimal self-coordination, a means including an emotion engine for recognizing emotions from the user's facial expressions and tone of voice, a means for generating prompts using a generative AI model, and a means for generating virtual try-on images based on the suggested content. This enables suggestions that reflect the user's individual requests and emotions, and the concrete visualization of the suggested content improves the user experience. Furthermore, automatic linking with various services and production software enables efficient operation.

[1337] "Means for capturing user images and requirements" refers to means for obtaining information from the user using a camera, microphone, or input interface.

[1338] "Prompt design methods" are methods that use acquired information to suggest youthfulness, optimal products and purchasing methods, beauty and hairstyles, health maintenance methods, clothing, training methods, etc.

[1339] "Means for realizing automatic integration with each service and production software" refers to a means for automatically purchasing products or reserving services based on the proposed content.

[1340] The "means for realizing optimal self-coordination" is a means for creating a specific coordination based on the proposed content and presenting it to the user.

[1341] "Means including an emotion engine" refers to means that has the function of recognizing emotions from the user's facial expressions and tone of voice.

[1342] "Means for generating prompts using a generative AI model" refers to means for generating prompts using a generative AI model based on the analysis results.

[1343] The "means for generating a virtual try-on image" is a means for generating an image in which the user virtually tries on clothes based on the proposed content.

[1344] The system of the present invention includes a means for capturing a user's image and requirements, a prompt design means, a means for automatically linking with each service and production software, a means for realizing optimal self-coordination, a means including an emotion engine, a means for generating prompts using a generative AI model, and a means for generating virtual try-on images.

[1345] A way to capture user images and requirements

[1346] Users take a photo of themselves using the camera on their smartphone or computer, and then speak their request into the microphone. This data is temporarily stored on the device and then sent to the server.

[1347] Sending and storing information

[1348] The device sends the acquired image data and audio data to the server. The server stores the received data in a database. For example, image data is stored in the "user_images" table, and audio data is stored in the "user_requests" table.

[1349] Analysis of information

[1350] The server uses image analysis algorithms to estimate the user's age and gender. For example, it uses OpenCV for facial recognition and an age estimation model to estimate age. It also uses voice recognition software (e.g., Google Cloud Speech-to-Text) to convert voice data into text. It also uses an emotion engine (e.g., Microsoft Azure Emotion API) to recognize emotions from the user's facial expressions and tone of voice.

[1351] Prompt Generation

[1352] The server generates prompts using a generative AI model (e.g., GPT-4) based on the analysis results. For example, it generates a prompt such as, "For a smiling woman in her late 30s, suggest casual clothing that emphasizes youthfulness."

[1353] Service collaboration

[1354] The server then connects with various services based on the generated prompts. For example, it uses an online shop's API to search for suggested outfits and complete the purchase process. It also connects with a hair salon's reservation system to book the most suitable salon for the user.

[1355] Coordination details

[1356] The server then creates a specific self-coordinated outfit based on the suggested content, for example, by generating an image of the user virtually trying on the suggested outfit. For this purpose, it uses image processing software.

[1357] Presentation of results

[1358] The server then sends the generated virtual try-on images and the results of the purchase process to the device. The user can then check the outfit on their smartphone or computer screen. For example, the device will display a notification, and when the user opens the app, the virtual try-on images and a link to purchase will be displayed.

[1359] Examples of specific examples and prompts

[1360] As a concrete example, consider the case where a user takes a photo of themselves with their smartphone camera and then voice-inputs, "Please suggest some clothes that will make me look younger." The system estimates the user's age and gender, and uses an emotion engine to recognize that the user is smiling. Based on this, the system suggests casual clothes that emphasize youthfulness, automatically searches for the clothes in an online shop, and completes the purchase process. It also generates virtual try-on images and presents them to the user.

[1361] Example prompt sentence:

[1362] "Based on the user's image and voice input, please suggest casual clothing that emphasizes youthfulness. The user is smiling, in their late 30s, and female. Search for the suggested outfits in an online shop, complete the purchase process, and generate and present a virtual try-on image."

[1363] In this way, the system of the present invention collects and analyzes user information, generates prompts, and works with various services to provide optimal self-coordination.

[1364] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1365] Step 1:

[1366] Users take a photo of themselves using the camera on their smartphone or computer, and then input their request by voice using the microphone. The input data consists of image data and voice data, which are temporarily stored on the device.

[1367] Step 2:

[1368] The device sends the saved image data and audio data to the server. The server saves the received data in a database. For example, image data is saved in the "user_images" table, and audio data is saved in the "user_requests" table. The input is image data and audio data, and the output is saved in the database.

[1369] Step 3:

[1370] The server uses an image analysis algorithm to estimate the user's age and gender. For example, it uses OpenCV for facial recognition and estimates age using an age estimation model. The input is image data, and the output is the estimated age and gender. It also uses speech recognition software (e.g., Google Cloud Speech-to-Text) to convert voice data into text. The input is voice data, and the output is text data. It then uses an emotion engine (e.g., Microsoft Azure Emotion API) to recognize emotions from the user's facial expressions and tone of voice. The input is image data and voice data, and the output is the recognized emotion.

[1371] Step 4:

[1372] The server generates a prompt using a generative AI model (e.g., GPT-4) based on the analysis results. For example, it generates a prompt such as "Suggest casual clothing that emphasizes youthfulness to a smiling woman in her late 30s." The input is the estimated age, gender, emotion, and text data, and the output is the generated prompt.

[1373] Step 5:

[1374] The server interacts with various services based on the generated prompt. For example, it uses an online shop's API to search for suggested outfits and then completes the purchase process. The input is the generated prompt, and the output is the completion of the purchase process. It also interacts with a hair salon reservation system to make a reservation at the most suitable hair salon for the user. The input is the generated prompt, and the output is the completion of the reservation.

[1375] Step 6:

[1376] The server creates a specific self-coordinated outfit based on the suggested content. For example, it generates an image of the user virtually trying on the suggested outfit. For this purpose, it uses image processing software. The input is the user's image data and the suggested outfit data, and the output is a virtual try-on image.

[1377] Step 7:

[1378] The server sends the generated virtual try-on images and the results of the purchase process to the device. The user can then check the outfit on their smartphone or computer screen. The input is the virtual try-on images and the results of the purchase process, and the output is what is presented to the user. For example, the device displays a notification, and when the user opens the app, the virtual try-on images and a link to purchase are displayed.

[1379] (Application example 1)

[1380] Next, a description will be given of Application Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1381] Conventional self-coordination systems were unable to provide suggestions that took user emotions into consideration, and were therefore unable to fully increase user satisfaction. Furthermore, they lacked the ability to purchase or reserve suggested items on the spot, resulting in low user convenience. This limited the user's shopping experience and missed monetization opportunities.

[1382] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1383] In this invention, the server includes a means for capturing a user's image and requirements, a prompt design means for suggesting youthfulness, optimal products and purchasing methods, beauty and hairstyles, health maintenance methods, clothing, training methods, etc., a means for automatically linking with each service and production software, a means for generating revenue by realizing optimal self-coordination, a means for using an emotion engine that recognizes the user's emotions, and a means for purchasing or reserving suggested items on the spot. This makes it possible to suggest optimal self-coordination that takes the user's emotions into consideration, and to immediately purchase or reserve suggested items.

[1384] "Means for capturing user images and requirements" refers to means for obtaining information from the user using a camera, microphone, input interface, etc.

[1385] "Prompt design methods" are methods that use acquired information to suggest youthfulness, optimal products and purchasing methods, beauty and hairstyles, health maintenance methods, clothing, training methods, etc.

[1386] "Means for realizing automatic integration with each service and production software" refers to a means for automatically purchasing products or reserving services based on the proposed content.

[1387] "A means of generating revenue by realizing optimal self-coordination" is a means of generating revenue by creating specific coordination based on the proposed content and presenting it to users.

[1388] "Means using an emotion engine to recognize user emotions" refers to means using an engine to recognize emotions from the user's facial expressions and tone of voice.

[1389] "Means for instantly purchasing or reserving the proposed item" means means for instantly purchasing or reserving the proposed item.

[1390] The system for implementing this invention includes means for capturing a user's image and requirements, prompt design means, means for realizing automatic linkage with each service and production software, means for realizing optimal self-coordination, means for using an emotion engine that recognizes the user's emotions, and means for purchasing or reserving suggested items on the spot.

[1391] System program processing explanation

[1392] Hardware and Software

[1393] Hardware: Smartphone built-in camera and microphone

[1394] software:

[1395] OpenCV: Image capture and processing

[1396] EmotionRecognizer: Emotion Recognition Model

[1397] FashionRecommender: A model for suggesting fashion coordination

[1398] ShoppingAPI: Product purchase and reservation API

[1399] Processing flow

[1400] 1. A way to capture user images and requirements:

[1401] It uses the smartphone's camera and microphone to capture the user's image and voice, thereby obtaining information about the user's appearance and requirements from their voice.

[1402] 2. Prompt design methods:

[1403] Based on the captured image and voice information of the user, a generative AI model is used to generate prompts that suggest youthfulness, optimal products and purchasing methods, beauty and hairstyles, health maintenance methods, clothing, and training methods.

[1404] 3. Using the Emotion Engine:

[1405] The captured image and audio are input into EmotionRecognizer to recognize the user's emotions. For example, if the user is smiling, it is recognized as "joy," if they are frowning, it is recognized as "anger," and if the tone of their voice is low, it is recognized as "sadness."

[1406] 4. Means for realizing automatic integration with various services and production software:

[1407] Based on the proposals generated by the prompt design tool, the system works in conjunction with each service and production software to automatically purchase products and reserve services.

[1408] 5. Ways to realize optimal self-coordination:

[1409] FashionRecommender is used to suggest optimal fashion coordination based on the user's emotions and requirements. The suggested coordination is visually presented to the user.

[1410] 6. How to purchase / reserve the proposed items on the spot:

[1411] The suggested items can be purchased or reserved on the spot using the Shopping API, allowing users to get the suggested items instantly.

[1412] Examples of specific examples and prompts

[1413] For example, here is a prompt for a user who prefers a casual style and wants blue items:

[1414] Example prompt sentence:

[1415] Capture a user's image and recognize their emotions. They prefer casual style and want blue items. Suggest the perfect outfit and purchase / reserve the suggested items on the spot.

[1416] In this way, it becomes possible to suggest optimal self-coordination ideas that take into account the user's emotions, and to instantly purchase or reserve the suggested items.

[1417] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1418] Step 1:

[1419] A user captures their own image and voice using the smartphone's camera and microphone.

[1420] Input: User image and voice

[1421] Output: Captured image and audio data

[1422] Specific actions: The user launches the camera app on their smartphone, points their face at the camera to take a picture, and speaks their requirements into the microphone.

[1423] Step 2:

[1424] The server receives the captured image and audio data and analyzes the images using image processing software (OpenCV).

[1425] Input: Captured image and audio data

[1426] Output: Analyzed image and audio information

[1427] Specific operation: The server inputs image data into OpenCV to perform facial recognition and feature extraction, and converts voice data into text.

[1428] Step 3:

[1429] The server uses EmotionRecognizer to recognize the user's emotions from the analyzed image information.

[1430] Input: Parsed image information

[1431] Output: Recognized emotion data

[1432] Specific operation: The server inputs image information into EmotionRecognizer, which estimates the user's emotion from their facial expression. For example, a smile indicates "joy," while a furrowed brow indicates "anger."

[1433] Step 4:

[1434] The server uses the generative AI model to generate prompts that provide optimal suggestions based on the user's emotions and requirements.

[1435] Input: Recognized emotion data and audio information (textual requirements)

[1436] Output: Generated prompt statement

[1437] Specific operation: The server inputs the emotion data and requirements into the generative AI model to generate a prompt sentence, such as "The user prefers a casual style and would like blue items. Please suggest the best fashion coordination."

[1438] Step 5:

[1439] The server uses FashionRecommender to suggest the best fashion coordination based on the generated prompt sentence.

[1440] Input: Generated prompt statement

[1441] Output: Proposed fashion outfit

[1442] Specific operation: The server inputs the prompt sentence into the FashionRecommender, which selects fashion items based on the user's requirements and emotions, such as a casual blue shirt and jeans.

[1443] Step 6:

[1444] The server presents the proposed fashion coordination to the user and uses the Shopping API to process the purchase or reservation on the spot.

[1445] Input: Proposed fashion outfit

[1446] Output: Purchased / reserved items

[1447] Specific operation: The server displays the suggested items on the user's smartphone, and when the user selects to purchase or reserve, the purchase / reservation procedure is completed through the Shopping API.

[1448] Example 2

[1449] Next, a description will be given of Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1450] In conventional systems, even if users input their own images and requirements, appropriate suggestions, automatic purchasing procedures, and suggestions based on emotions were not made, resulting in low user satisfaction and the inability to achieve efficient self-coordination. In addition, since suggestions were not made taking into account the user's emotions, the system was unable to fully meet the user's needs.

[1451] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1452] In this invention, the server includes a means for capturing a user's image and requirements, a prompt design means for suggesting youthfulness, optimal products and purchasing methods, beauty and hairstyles, health maintenance methods, clothing, training methods, etc., a means for realizing automatic linkage with each service and production software, a means for generating revenue by realizing optimal self-coordination, a means for using an emotion engine that recognizes the user's emotions, and a means for making suggestions based on emotions. This enables appropriate suggestions based on the user's input information, automatic purchasing procedures, and suggestions based on emotions, thereby increasing user satisfaction and enabling efficient self-coordination.

[1453] The "means for capturing user images and requirements" is an interface through which a user can input their own image and specific requirements into the system and transmit that information to the server.

[1454] The "prompt design tool" is a function that suggests youthfulness, optimal products and purchasing methods, beauty and hairstyles, health maintenance methods, clothing, training methods, etc. based on the information entered by the user.

[1455] "Automatic linking means" is a function that automatically links with each service or production software to purchase products or reserve services based on the user's selection.

[1456] "Means for realizing optimal self-coordination" is a function for creating specific coordination based on selected products and services and the user's image, and presenting it to the user.

[1457] The "emotion engine" is an analytical engine that recognizes the user's emotions and makes suggestions based on those emotions.

[1458] "Means for making suggestions based on emotions" is a function that suggests optimal products and services, beauty and hairstyles, health maintenance methods, clothing, training methods, etc. based on the user's emotions recognized by the emotion engine.

[1459] This invention is a system that allows users to input their own image and requirements, and then suggests optimal products and services, beauty and hairstyles, health maintenance methods, clothing, training methods, etc. for maintaining a youthful appearance, and also automatically handles purchase procedures and service reservations. Specific embodiments of this system are described below.

[1460] System configuration

[1461] The system consists of the following main components:

[1462] 1. A way to capture user images and requirements

[1463] 2. Prompt Design Methods

[1464] 3. Automatic linking methods

[1465] 4. Means of realizing optimal self-coordination

[1466] 5. Emotion Engine

[1467] 6. Emotion-based suggestions

[1468] Hardware and Software

[1469] Terminal: The device through which the user enters images and requirements (e.g. smartphone, PC).

[1470] Server: A central processing unit for data processing and proposal generation.

[1471] Generative AI model: An artificial intelligence model for generating prompts based on user requirements.

[1472] Emotion Engine: An analysis engine for recognizing user emotions.

[1473] Image Processing Software: Software that superimposes selected products and services onto a user's image.

[1474] Data processing and calculation

[1475] 1. Obtaining user input information

[1476] The user uses the device to take a photo of themselves and enters the requirement that they want to look "youthful."

[1477] The terminal sends this information to the server.

[1478] 2. Proposal by prompt design method

[1479] The server uses a generative AI model to generate prompts based on the user's input.

[1480] Based on the generated prompts, it suggests youthful clothing, hairstyles, and ways to stay healthy.

[1481] 3. User Choice

[1482] The user uses the device to select the outfit they like from the suggestions.

[1483] The device sends the user's selection to the server.

[1484] 4. Purchasing procedures using automated linking methods

[1485] The server uses the online shop's API to complete the purchase of the selected outfit.

[1486] 5. Realizing optimal self-coordination

[1487] The server uses image processing software to combine the selected outfit with the user's image and sends the result to the device.

[1488] 6. Emotion Recognition by Emotion Engine

[1489] The server uses an emotion engine to recognize the user's emotion.

[1490] 7. Emotion-based recommendations

[1491] The server uses the generative AI model again to generate emotionally appropriate prompts based on the emotions recognized by the emotion engine.

[1492] Specific examples

[1493] A user takes a picture of themselves with their smartphone and enters "I want to look youthful." The device sends this information to a server, which uses a generative AI model to suggest youthful, casual clothing. The user selects a red jacket and the device sends the selection to the server. The server uses an online shop's API to complete the purchase of the red jacket and uses the Adobe Photoshop API to superimpose the red jacket onto the user's image. If the user expresses the emotion "joy," the server uses an emotion engine to recognize the emotion and uses a generative AI model to suggest bright-colored clothing and exercise methods that match the happy mood.

[1494] Prompt Sentence Examples

[1495] "The user inputs their own image and their desired youthful appearance. Please suggest youthful clothing, hairstyles, and health maintenance methods, automatically complete the procedure for purchasing the selected clothing from an online shop, and suggest the best self-coordination."

[1496] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1497] Step 1:

[1498] The user enters their image and requirements.

[1499] Input: User image, youthfulness requirement

[1500] Specific actions: The user takes a picture using a smartphone or PC and enters "I want to look younger" into the text box.

[1501] Output: The user's image and requirements are saved on the device and sent to the server.

[1502] Step 2:

[1503] The server generates the suggestions using a prompt design tool.

[1504] Input: User image, youthfulness requirement

[1505] What it does: The server uses the generative AI model to generate prompts based on the user's requirements.

[1506] Data processing: A generative AI model analyzes the user's image and requirements to generate suggestions for youthful clothing, hairstyles, and health maintenance methods.

[1507] Output: The suggestions are generated in a list format and sent to the device.

[1508] Step 3:

[1509] The user selects the outfit they like from the suggestions.

[1510] Input: A list of suggested outfits

[1511] What happens: The user clicks to select an item from the on-screen suggestion list.

[1512] Output: The user's selection is saved on the device and sent to the server.

[1513] Step 4:

[1514] The server performs the purchase procedure using an automatic linking means.

[1515] Input: User's chosen outfit

[1516] Specific operation: The server calls the online shop's API and sends the purchase information for the selected item.

[1517] Data processing: The server sends the information necessary for the purchase procedure to the online shop and completes the purchase.

[1518] Output: Purchase confirmation information is stored on the server and sent to the device.

[1519] Step 5:

[1520] The server embodies the optimal self-coordination.

[1521] Input: User image, selected outfit

[1522] What it does: The server uses image processing software to composite the selected outfit onto the user's image.

[1523] Data processing: Image processing software combines the user's image with the selected outfit to generate an outfit image.

[1524] Output: The generated coordinated image is sent to the device.

[1525] Step 6:

[1526] The server uses an emotion engine to recognize the user's emotions.

[1527] Input: User facial expressions and text input

[1528] Specific operation: The server uses an emotion engine to analyze emotions from the user's facial expressions and text.

[1529] Data processing: The emotion engine generates user emotion data.

[1530] Output: The recognized emotion data is stored on the server.

[1531] Step 7:

[1532] The server makes suggestions based on emotions.

[1533] Input: Recognized emotion data

[1534] Specific operation: The server uses the generative AI model again based on the emotion data to generate a prompt appropriate to the emotion.

[1535] Data processing: A generative AI model analyzes the emotion data and generates emotion-based recommendations.

[1536] Output: Sentiment-based suggestions are sent to the device.

[1537] Step 8:

[1538] The server uses an automatic linking method to make purchases and reservations.

[1539] Input: Suggestions based on user choices and sentiment data

[1540] Specific operation: The server calls the API of the online shop or service provider and sends purchase or reservation information.

[1541] Data processing: The server sends the information necessary for the purchase or reservation process to the online shop or service provider to complete the process.

[1542] Output: Purchase and reservation confirmation information is stored on the server and sent to the device.

[1543] Step 9:

[1544] The server embodies the optimal self-coordination.

[1545] Input: User image, selected products and services, emotional data

[1546] What it does: The server uses image processing software to composite the selected products and services onto the user's image.

[1547] Data processing: Image processing software combines the user's image with the selected products and services to generate a coordinated image.

[1548] Output: The generated coordinated image is sent to the device.

[1549] (Application example 2)

[1550] Next, a description will be given of Application Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1551] Conventional online shopping systems have had the problem of being difficult to recommend optimal products and complete purchasing procedures based on the user's individual needs and emotions. Furthermore, users cannot receive specific outfit suggestions based on their own images or emotions, making it difficult to stimulate purchase motivation. Furthermore, there is a lack of a way to automatically purchase the products selected by the user, making the system less convenient.

[1552] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1553] In this invention, the server includes means for capturing a user's image and requirements, prompt design means for suggesting youthfulness, optimal products and purchasing methods, beauty and hairstyles, health maintenance methods, clothing, training methods, etc., means for automatically linking with each service and production software, means for generating revenue by realizing optimal self-coordination, means for analyzing the user's emotions using an emotion recognition engine, means for generating suggestions based on the user's image features and emotions using a generative AI model, and means for automatically connecting to an online shop selling the suggested products and carrying out the purchase procedure. This enables optimal product suggestions and purchase procedures based on the user's individual requests and emotions.

[1554] "Means for capturing user images and requirements" refers to an interface through which a user can input their own image and specific requests into the system.

[1555] "Prompt design tools" are functions that suggest youthful clothing, hairstyles, and health maintenance methods based on the user's images, requirements, and emotions.

[1556] "Means for realizing automatic linkage" refers to a function that automatically links with external systems or online shops that offer suggested products or services, and allows for purchase procedures and reservations.

[1557] "Means for realizing optimal self-coordination" is a function for generating specific coordination based on selected products and services and the user's images and emotions, and presenting it to the user.

[1558] An "emotion recognition engine" is a technology that analyzes emotions from a user's images and voice, and recognizes emotional states such as "joy" and "sadness."

[1559] A "generative AI model" is an artificial intelligence technology that generates optimal suggestions based on the user's image features and emotions.

[1560] "Means for automatically connecting to an online store and completing the purchase process" is a function that automatically connects to an online store that sells the product selected by the user and automates the purchase process.

[1561] The system for implementing this invention includes means for capturing a user's image and requirements, prompt design means, means for realizing automatic coordination, means for realizing optimal self-coordination, an emotion recognition engine, a generative AI model, and means for automatically connecting to an online shop and carrying out the purchase process.

[1562] System configuration

[1563] 1. A way to capture user images and requirements

[1564] Users use a smartphone app to upload an image of themselves and enter their desired youthful appearance.

[1565] Use an image recognition library (e.g., OpenCV) to analyze the user's facial and clothing features.

[1566] 2. Prompt Design Methods

[1567] Based on the user's image characteristics and requirements, the app suggests youthful clothing, hairstyles, and health maintenance methods.

[1568] Use a generative AI model (e.g., OpenAI GPT-4) to generate suggestions.

[1569] Example: "Based on the user's image and emotions, suggest youthful clothing, hairstyles, and ways to stay healthy."

[1570] 3. Emotion Recognition Engine

[1571] Analyze user emotions using an emotion recognition engine (e.g. Microsoft Azure Emotion API).

[1572] Based on the analysis results, the user's emotions are classified as "joy" or "sadness," etc.

[1573] 4. Generative AI Models

[1574] Using a generative AI model, suggestions are generated based on the user's image features and emotions.

[1575] Example: "Based on the user's image and emotions, suggest youthful clothing, hairstyles, and ways to stay healthy."

[1576] 5. Means of realizing automatic collaboration

[1577] It automatically connects with external systems and online shops that offer suggested products and services to carry out purchase procedures and reservations.

[1578] Use online store APIs to automate the checkout process.

[1579] 6. Means of realizing optimal self-coordination

[1580] Based on the selected products and services, the user's images, and their emotions, a specific coordination is generated and presented to the user.

[1581] Specific examples

[1582] The user uploads a photo of themselves to a smartphone app and enters the phrase "I want to look youthful." The emotion recognition engine recognizes the emotion of "joy," and the generative AI model generates suggestions for "light-colored casual clothing and jogging that matches the joyful mood." The user selects the suggested outfit, and the app automatically connects to the online shop to complete the purchase.

[1583] In this way, optimal product suggestions and purchasing procedures can be made based on the user's individual needs and emotions.

[1584] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1585] Step 1:

[1586] Users upload their own images to a smartphone app and input their desired youthful appearance.

[1587] Input: User image, youthfulness requirement

[1588] Output: Image data, requirements data

[1589] What it does: The user takes a picture of themselves using the smartphone's camera and enters their requirements into the app's input form, which then sends the data to the server.

[1590] Step 2:

[1591] The server uses an image recognition library (e.g., OpenCV) to analyze the user's facial and clothing features.

[1592] Input: Image data

[1593] Output: Face feature data, clothing feature data

[1594] Specific operation: The server analyzes the received image data and extracts features such as facial contours, clothing color, and shape. These feature data are used for subsequent processing.

[1595] Step 3:

[1596] The server uses an emotion recognition engine (e.g., Microsoft Azure Emotion API) to analyze the user's emotions.

[1597] Input: Facial feature data

[1598] Output: Emotion data (e.g., happiness, sadness)

[1599] Specific operation: The server inputs facial feature data into an emotion recognition engine to analyze the user's emotional state. As a result of the analysis, the user's emotion is classified as "happiness" or "sadness," etc.

[1600] Step 4:

[1601] The server uses a generative AI model (e.g., OpenAI GPT-4) to generate suggestions based on the user's image features and emotions.

[1602] Input: Facial feature data, clothing feature data, emotion data, requirement data

[1603] Output: Suggested data (e.g., youthful clothing, hairstyles, and health maintenance methods)

[1604] Specific operation: The server inputs the user's image features, emotions, and requirements into the generative AI model to generate youthful suggestions, including clothing, hairstyles, and health maintenance methods suitable for the user.

[1605] Step 5:

[1606] The server automatically connects with external systems and online shops that offer suggested products and services, and handles purchase procedures and reservations.

[1607] Input: Proposed data, user selected data

[1608] Output: Purchase procedure data, reservation data

[1609] Specific operation: When the user selects a suggested product or service, the server automatically uses the online shop API to purchase the selected product and, if necessary, reserve the service.

[1610] Step 6:

[1611] The server generates specific coordination based on the selected products and services and the user's images and emotions, and presents it to the user.

[1612] Input: Selection data, facial feature data, clothing feature data, emotion data

[1613] Output: Coordinate data

[1614] Specific operation: The server generates a specific outfit based on the selected products and services, the user's image features, and their emotions. The generated outfit is visually presented to the user, allowing them to achieve optimal self-coordination.

[1615] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1616] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1617] Another example of generative AI is Gemini (internet search engine). <url: https: gemini.google.com ?hl="ja">) are listed.

[1618] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[1619] [Fourth embodiment]

[1620] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[1621] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1622] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1623] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[1624] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1625] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1626] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1627] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[1628] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1629] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1630] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1631] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1632] Next, the specific processing by the specific processing unit 290 of the data processing device 12 will be described.

[1633] "Example 1"

[1634] The system of the present invention includes a means for capturing a user's image and requirements, a prompt design means, a means for automatically linking with various services and production software, and a means for realizing optimal self-coordination. The means for capturing a user's image and requirements acquires information from the user using a camera, microphone, input interface, etc. The prompt design means suggests youthfulness, optimal products and purchasing methods, beauty and hairstyles, health maintenance methods, clothing, training methods, etc. based on the acquired information. The means for automatically linking with various services and production software automatically purchases products and reserves services based on the suggested content. The means for realizing optimal self-coordination creates specific coordination based on the suggested content and presents it to the user.

[1635] "Example 2"

[1636] As a specific example, a user inputs an image of themselves and their desire to look youthful into the system. Based on this information, the prompt design means suggests youthful clothing, hairstyles, and health maintenance methods. When the user selects an outfit they like from the suggestions, the automatic linking means automatically connects to an online shop selling the selected outfit and completes the purchase process. Furthermore, the means for realizing optimal self-coordination creates a specific outfit based on the selected outfit and the user's image and presents it to the user.

[1637] The processing flow of each embodiment will be described below.

[1638] "Example 1"

[1639] Step 1: The user inputs their own image and requirements (e.g., a youthful appearance) into the system. This input is done using a camera, microphone, input interface, etc.

[1640] Step 2: The prompt design tool uses the input information to suggest youthful clothing, hairstyles, and health maintenance methods. These suggestions are based on the user's age, gender, body type, preferences, and other information.

[1641] Step 3: The user selects the outfit they like from the suggestions.

[1642] Step 4: The automatic linking means automatically connects to the online shop selling the selected clothing and carries out the purchase procedure.

[1643] Step 5: The means for realizing the optimal self-coordination creates a specific coordination based on the selected outfit and the user's image and presents it to the user.

[1644] "Example 2"

[1645] Step 1: The user inputs their image and requirements (e.g., health maintenance) into the system. This input is done using a camera, microphone, input interface, etc.

[1646] Step 2: The prompt design tool uses the input information to suggest healthy diets and exercise methods. These suggestions are based on the user's age, gender, body type, preferences, and other information.

[1647] Step 3: The user selects their preferred diet and exercise plan from the suggestions.

[1648] Step 4: An automatic linking means automatically makes reservations at restaurants that serve the selected meals and at training gyms that are suitable for the selected exercise method.

[1649] Step 5: The means for realizing optimal self-coordination creates a specific lifestyle based on the selected diet and exercise method and the user's image, and presents it to the user.

[1650] Example 1

[1651] Next, a description will be given of Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1652] Conventional self-coordination systems struggled to provide suggestions based on the individual requirements and characteristics of users, and lacked automatic coordination to translate the suggestions into concrete actions. As a result, users had to perform many manual operations, resulting in low convenience. Furthermore, the suggestions were not sufficiently detailed or visually presented, and there was a lack of ingenuity to make them easy for users to understand. This led to issues such as lower user satisfaction and a decrease in the frequency of system use.

[1653] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1654] In this invention, the server includes a means for capturing a user's image and requirements, a prompt design means for suggesting youthfulness, optimal products and purchasing methods, beauty and hairstyles, health maintenance methods, clothing, and training methods, a means for automatically linking with various services and production software, a means for generating revenue by realizing optimal self-coordination plans, a means for analyzing acquired image data using facial recognition technology, a means for converting voice data into text using speech recognition technology, a means for generating prompts using a generative AI model, a means for creating specific self-coordination plans based on the suggested content, and a means for presenting the generated self-coordination plans to the user. This enables suggestions based on the user's individual requirements and characteristics, realizes automatic linkage to translate the suggested content into concrete actions, and improves user convenience. Furthermore, the concreteness and visual presentation of the suggested content improves user understanding, improving user satisfaction and increasing system usage frequency.

[1655] "Means for capturing user images and requirements" refers to devices and software for obtaining image data, audio data, and text data from users using a camera, microphone, input interface, etc.

[1656] "Prompt design tools" are algorithms and software that use acquired user information to suggest youthfulness, optimal products and purchasing methods, beauty and hairstyles, health maintenance methods, clothing, training methods, etc.

[1657] "Means for realizing automatic integration with each service and production software" refers to an interface or API that automatically connects with external services such as online shops and reservation systems based on the proposed content, allowing products to be purchased or services to be reserved.

[1658] "Means for generating revenue by realizing optimal self-coordination" refers to a method or system for generating revenue by creating specific self-coordination based on proposed content and presenting it to users, thereby encouraging them to purchase products or use services.

[1659] "Means of analysis using facial recognition technology" refers to algorithms or software for extracting a user's age, gender, facial features, etc. from the acquired image data.

[1660] "Means of converting voice data into text using speech recognition technology" refers to algorithms or software that convert acquired voice data into text data and understand the user's requirements.

[1661] "Means for generating prompts using a generative AI model" refers to an artificial intelligence model or software that generates prompts containing suggestions based on the analysis results in accordance with the user's requirements.

[1662] The "means for creating specific self-coordination" refers to image processing technology and software that synthesizes clothing and hairstyles with the user's image based on the proposed content, generating a finished image.

[1663] The "means for presenting the generated self-coordinated image to the user" refers to an interface or software for transmitting the generated self-coordinated image to the user's terminal and displaying it.

[1664] This invention is a system that includes a means for capturing a user's image and requirements, a prompt design means, a means for automatically linking with various services and production software, and a means for realizing optimal self-coordination. Specific embodiments of this system are described below.

[1665] Retrieving User Information

[1666] Users access the system using a device (such as a smartphone or PC). They take a photo of themselves using the device's camera and input their requirements by voice using the microphone. Requirements can also be input as text using a keyboard or touch screen. For example, a user can take a photo of themselves using the smartphone's camera and say, "I'd like suggestions for a youthful style" using the microphone.

[1667] Sending data

[1668] The device sends the acquired image data, audio data, and text data to the server via an internet connection, such as Wi-Fi or mobile data.

[1669] Data analysis

[1670] The server analyzes the received data. For image data, it uses facial recognition technology (e.g., OpenCV) to extract the user's age, gender, and facial features. For voice data, it uses speech recognition technology (e.g., Google Speech-to-Text API) to convert it into text and understand the user's requirements. For text data, it uses natural language processing technology (e.g., spaCy) to analyze the requirements.

[1671] Generate prompt statement

[1672] The server generates a prompt using a generative AI model (e.g., GPT-4) based on the analysis results. For example, if a user inputs "Please suggest a youthful style," the server generates the prompt "Please suggest a youthful casual style for women in their 30s."

[1673] Service collaboration

[1674] The server then connects to various services and production software based on the generated prompt text. For example, it uses an online shop's API to search for suggested outfits and proceed with the purchase. It also connects with a hair salon's reservation system to make a reservation for a hairstyle.

[1675] Coordination details

[1676] The server then creates a specific self-coordinated outfit based on the suggestions. For example, it uses image processing software to combine the suggested clothing and hairstyle with the user's photo to create a finished image.

[1677] Presentation of results

[1678] The server sends the generated self-coordinated image to the terminal, which displays the results to the user. The user can then review the proposed coordinated outfit and make purchases or reservations as necessary.

[1679] In this way, users can easily find the perfect outfit for themselves. A concrete example of a prompt sentence is, "Please suggest a youthful, casual style for a woman in her 30s." This system makes suggestions based on the user's individual requirements and characteristics, and automatically links the suggestions to concrete actions. This improves user convenience and enhances user satisfaction by making the suggestions more concrete and visually presenting them.

[1680] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1681] Step 1:

[1682] Users access the system using a terminal. They take their own photo with the terminal's camera and input their requirements by voice using the microphone. They can also input requirements in text using a keyboard or touch screen. Input data can be image data, voice data, or text data.

[1683] Step 2:

[1684] The device transmits the acquired image data, voice data, and text data to the server via an Internet connection. The input is the data acquired from the user, and the output is the data sent to the server.

[1685] Step 3:

[1686] The server analyzes the received image data using facial recognition technology (e.g., OpenCV) to extract the user's age, gender, and facial features. The input is image data, and the output is the user's feature data as the analysis result.

[1687] Step 4:

[1688] The server converts the received voice data into text using speech recognition technology (e.g., Google Speech-to-Text API) to understand the user's requirements. The input is voice data, and the output is text data.

[1689] Step 5:

[1690] The server analyzes the received text data using natural language processing technology (e.g., spaCy) to understand the user's requirements. The input is text data, and the output is requirement data as the analysis result.

[1691] Step 6:

[1692] The server generates a prompt sentence using a generative AI model (e.g., GPT-4) based on the analysis results. For example, if a user inputs "Please suggest a youthful style," the server generates the prompt sentence "Please suggest a youthful casual style for women in their 30s." The input is the analysis result, and the output is the prompt sentence.

[1693] Step 7:

[1694] The server interacts with various services and production software based on the generated prompt text. For example, it uses an online shop's API to search for suggested outfits and proceed with the purchase process. It also interacts with a hair salon's reservation system to make a hairstyle reservation. The input is the prompt text, and the output is the result of the service interaction.

[1695] Step 8:

[1696] The server creates a specific self-coordinated outfit based on the proposed content. For example, it uses image processing software to combine the suggested clothing and hairstyle with the user's photo to generate a finished image. The input is the proposed content and the user's photo, and the output is the finished image.

[1697] Step 9:

[1698] The server sends the generated self-coordinated image to the terminal, which displays the result to the user. The input is the completed image, and the output is the display to the user. The user can review the proposed coordinated image and make purchases or reservations as necessary.

[1699] (Application example 1)

[1700] Next, a description will be given of Application Example 1 of Embodiment Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1701] Conventional self-coordination systems have the problem that it is difficult for users to find the suggested products in physical stores, resulting in an inefficient shopping experience. Additionally, while there are systems that suggest optimal outfits based on the user's images and requirements, they lack the functionality to provide navigation information within physical stores, making it difficult for users to quickly find the suggested products.

[1702] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means. In this invention, the server includes a means for capturing a user's image and requirements, a prompt design means for suggesting youthfulness, optimal products and purchasing methods, beauty and hairstyles, health maintenance methods, clothing, and training methods, etc., a means for automatically linking with each service and production software, a means for generating revenue by realizing optimal self-coordination, and a means for providing navigation information for finding suggested products in a physical store. This allows the user to quickly find the suggested products in the physical store, making the shopping experience efficient and enjoyable.

[1703] "Means for capturing user images and requirements" refers to means for obtaining information from the user using a camera, microphone, input interface, etc.

[1704] "Prompt design methods" are methods that use acquired information to suggest youthfulness, optimal products and purchasing methods, beauty and hairstyles, health maintenance methods, clothing, training methods, etc.

[1705] "Means for realizing automatic integration with each service and production software" refers to a means for automatically purchasing products or reserving services based on the proposed content.

[1706] "A means of generating revenue by realizing optimal self-coordination" is a means of generating revenue by creating specific coordination based on the proposed content and presenting it to users.

[1707] The "means for providing navigation information for finding suggested products in a physical store" refers to a means for navigating so that suggested products can be easily found in a physical store.

[1708] As an embodiment of the present invention, the following system is constructed.

[1709] First, the smartphone's camera, microphone, and input interface are used to capture the user's image and requirements. The user takes a picture of themselves using the smartphone's camera and imports it into the app, which then inputs the user's image and requirements into the system.

[1710] Next, as a prompt design method, based on the acquired user image and requirements, the system suggests youthfulness, optimal products and purchasing methods, beauty and hairstyles, health maintenance methods, clothing, training methods, etc. For this, a generative AI model using Keras and TensorFlow is used. Specifically, the user's image is input into the AI ​​model, and a prompt is generated that suggests the optimal outfit.

[1711] Furthermore, as a means of achieving automatic integration with various services and produ...

Claims

1. means for capturing user information including at least one of an image and requirements of a user; means for recognizing an emotion of the user using an emotion engine; a prompt design means for making a suggestion related to self-coordination of the user based on the user information and the user's emotion; A means for implementing the proposal in cooperation with an external service; A system including:

2. The prompt design means makes the suggestion based on at least one of the user's age, sex, body type, and preference information. The system of claim 1 .

3. The implementing means automatically executes a transaction for the product or service related to the proposal based on the user's selection. The system of claim 1 .

Images