System

Abstract

A system is provided.SOLUTION: A system includes a means for a user to upload a face photo and a style photo, a means for the user to input a situation and a preference, a means for a terminal to transmit information of the face photo, the style photo, and the situation and the preference to a server, a means for the server to analyze the received information and generate a makeup, a hairstyle, and a coordinate using a generated AI model, a means for the server to transmit the generation result to the terminal, a means for the terminal to display the generation result to the user, a means for the user to make a reservation and pay based on the generation result, and a means for the server to process the reservation and the payment.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] Users face challenges such as the fact that it takes a lot of time and effort to find the makeup, hairstyle, and outfit that best suits them, that they often need expert advice due to their lack of knowledge about beauty and fashion, and that it is difficult to find time in their busy daily lives to visit a hair salon, beauty salon, or clothing store. [Means for solving the problem]

[0005] The present invention solves the above problem by providing a system including a means for a user to upload a face photo and a style photo, a means for the user to input the situation and preferences, a means for a terminal to send the face photo, the style photo, the situation and preference information to a server, a means for the server to analyze the received information and generate makeup, hairstyle and coordination using a generative AI model, a means for the server to send the generated results to the terminal, a means for the terminal to display the generated results to the user, a means for the user to make a reservation and make a payment based on the generated results, and a means for the server to process the reservation and payment.

[0006] "User" means an individual or corporation that uses this system.

[0007] A "face photo" is image data that shows the user's face.

[0008] A "style photo" is image data that shows the user's entire body or partial fashion style.

[0009] A "situation" is information that indicates the situation or scene related to the user's activities or events for that day.

[0010] "Preferences" refers to the user's personal preferences and desires regarding makeup, hairstyles, and fashion choices.

[0011] A "terminal" is a device used by a user, such as a smartphone, tablet, or PC.

[0012] A "server" is a central processing unit that receives and analyzes data sent by users.

[0013] A "generative AI model" is an algorithm that uses artificial intelligence technology to analyze user information and generate optimal makeup, hairstyles, and outfits.

[0014] "Generated results" are the makeup, hairstyle, and outfit suggestions that are output as a result of analysis by the generative AI model.

[0015] "Reservation and payment" refers to the process in which a user makes a reservation for a service at a beauty salon, esthetic salon, clothing store, etc. based on the proposed content and pays the fee.

[0016] A "business" is a store or company that provides services to users, such as a beauty salon, esthetic salon, or clothing store. [Brief explanation of the drawings]

[0017] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 3 is a sequence diagram showing a processing flow of the data processing system according to the first embodiment. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0018] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0019] First, the terms used in the following description will be explained.

[0020] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0021] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0022] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0023] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0024] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0025] [First embodiment]

[0026] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0027] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0028] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0029] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0030] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0031] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0032] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0033] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0034] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0035] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0036] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0037] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0038] System Overview

[0039] This invention is a system that allows users to receive recommendations for the best makeup, hairstyle, and outfits for themselves, and to make reservations and payments at beauty salons, esthetic salons, and clothing stores all in one place. The system begins when users upload photos of their face and style, and input the situation and preferences. A specific embodiment of this system is described in detail below.

[0040] Program processing flow

[0041] 1. User Input Processing

[0042] When a user launches the application, the device requests the user's account authentication information and prompts the user to log in. After logging in, the user uploads a face photo and a full-body photo. The user also inputs the situation of the day (e.g., date, work, party) and their personal preferences (e.g., natural makeup, casual style).

[0043] 2. Data transmission process

[0044] The device sends the facial photo, full-body photo, situation, and preference information acquired from the user to the server as a single data package. After the transmission is successful, the device notifies the user of the success.

[0045] 3. Data analysis and proposal generation process

[0046] The server analyzes the received data and inputs it into a generative AI model. The generative AI model extracts facial features (shape, skin tone, eyebrow shape, etc.) from the user's face photo and analyzes the user's body type and style from a full-body photo. It then generates optimal makeup, hairstyles, and outfits based on the situation and preferences. The generated suggestions are sent from the server to the device.

[0047] 4. Display of proposed results

[0048] The device analyzes the generated results received from the server and displays them in a format that is easy for the user to understand. The suggestions include the most suitable makeup, hairstyle, and outfit. The user checks the displayed suggestions and, if they like them, proceeds to the next step.

[0049] 5. Booking and payment processing

[0050] If the user likes the suggestions, they use the device's interface to search for nearby beauty salons, esthetic salons, and clothing stores, make a reservation, and pay. The device then sends the reservation information and payment data to the server, which relays it to the appropriate provider. The provider then sends a reservation confirmation to the server, which then processes the successful payment. Finally, the device displays a notification that the reservation and payment have been completed.

[0051] Specific examples

[0052] For example, consider the case where a user uploads a photo of their face and a full body in a situation where they want to know what kind of hairstyle is good for their date. If the user enters "I like casual styles," the system operates as follows:

[0053] 1. User input processing: The user uploads a photo of their face and a full-body photo, and enters the situation and preferences.

[0054] 2. Data transmission process: The device sends a face photo, a full-body photo, the situation, and preference information to the server.

[0055] 3. Data analysis and proposal generation process: The server analyzes the received information and proposes natural curly hair as a "casual hairstyle suitable for a date."

[0056] 4. Displaying the proposal results: The device displays the proposals for natural curly hair to the user.

[0057] 5. Booking and payment processing: The user decides to try the suggested hairstyle, searches for a nearby hair salon, makes a booking and makes payment.

[0058] In this way, users can easily find the makeup, hairstyle, and outfit that suits them, and can smoothly make reservations and make payments. This system provides optimal services to users who are interested in beauty and fashion but don't have much time.

[0059] The processing flow will be explained below.

[0060] Step 1:

[0061] The user launches the application. The device prompts the user for account authentication information, and the user enters the login information.

[0062] Step 2:

[0063] The user takes or uploads a photo of their face and a full-body photo, and then inputs the situation (e.g., date, work, party) and preferences (e.g., natural makeup, casual style).

[0064] Step 3:

[0065] The terminal transmits the face photo, the style photo, the situation, and the preference information to the server as a single data package. After the transmission is successful, the terminal notifies the user of the success.

[0066] Step 4:

[0067] The server inputs the received data into a generative AI model for analysis. The generative AI model extracts facial features (shape, skin tone, eyebrow shape, etc.) from the user's face photo and analyzes the user's body shape and style from a full-body photo.

[0068] Step 5:

[0069] The server generates the optimal makeup, hairstyle, and outfit based on the analysis results and the user's situation and preferences, and sends the generated results to the device.

[0070] Step 6:

[0071] The device analyzes the generated results received from the server and displays them in a format that is easy for the user to understand. The user checks the proposal and, if they like it, proceeds to the next step.

[0072] Step 7:

[0073] The user decides to make a reservation and make payment based on the proposed service. The terminal searches for nearby beauty salons, esthetic salons, and clothing stores and displays a list of businesses that can provide the proposed service.

[0074] Step 8:

[0075] The user selects a vendor and specifies a reservation date and time. After the selection and reservation information are confirmed, the terminal sends the reservation information and payment data to the server.

[0076] Step 9:

[0077] The server relays the reservation information to the merchant and receives confirmation of the reservation. The server processes the payment using PayPay and confirms the payment was successful.

[0078] Step 10:

[0079] The server sends a notification of completion of reservation and payment to the terminal. The terminal notifies the user that reservation and payment have been completed.

[0080] This series of processing flows allows users to easily and quickly receive suggestions for optimal makeup, hairstyles, and outfits, while also smoothly making reservations and making payments.

[0081] Example 1

[0082] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0083] Previously, to receive beauty and fashion recommendations, users had to search for multiple services and stores individually, make reservations, and pay. This was time-consuming and labor-intensive, making it difficult to receive optimal recommendations. Furthermore, technology to automatically generate recommendations tailored to a user's preferences and situation was insufficient, limiting the user experience.

[0084] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0085] In this invention, the server includes a means for inputting prompt text into the generative AI model to suggest makeup, hairstyles, and outfits, a means for the server to transmit the generated results to a terminal, and a means for the user to make reservations and payments based on the generated results. This allows users to receive optimal beauty and fashion suggestions simply by uploading photos of their face and style and inputting the situation and preferences. Furthermore, reservations and payments can be made all at once, significantly reducing the user's time and effort.

[0086] "User" refers to an individual or organization that uses the system.

[0087] "Terminal" refers to the device used by a User (e.g., smartphone, tablet, computer).

[0088] A "face photo" refers to an image of the user's face.

[0089] A "style photo" refers to an image of the user's entire body.

[0090] A "situation" refers to a specific event or purpose (e.g., date, work, party) entered by the user.

[0091] "Preferences" refers to the preferences regarding fashion and makeup input by the user (e.g., natural makeup, casual style).

[0092] "Server" refers to a computer system that processes, stores, and transmits data.

[0093] A "data package" refers to a collection of data including a face photo, style photo, situation, and preference information.

[0094] A "generative AI model" refers to an artificial intelligence algorithm that suggests makeup, hairstyles, and outfits based on input data.

[0095] A "prompt" refers to a specific instruction or request given to a generative AI model.

[0096] "Suggestion generation" refers to the process by which the generative AI model generates the best makeup, hairstyle, and outfit for the user based on the analysis results.

[0097] "Reservation" refers to the act of a user setting a date and time for using services at a beauty salon, esthetic salon, or clothing store based on the content suggested by the user.

[0098] "Payment" refers to the act of paying monetary consideration for the use of a service.

[0099] "Businesses" refer to service providers such as hair salons, beauty salons, and clothing stores.

[0100] "Generated results" refers to the makeup, hairstyle, and outfit suggestions generated by the generative AI model.

[0101] "Relay" refers to the act of the server forwarding information received from the user to the appropriate provider.

[0102] This invention is a system that allows users to receive suggestions for the best makeup, hairstyle, and outfits for themselves, and to make reservations and payments at beauty salons, esthetic salons, and clothing stores all in one place. This system is realized mainly using a terminal, a server, and a generative AI model.

[0103] Specific program description

[0104] User Input Processing

[0105] A user launches the application and uses their device to enter their account credentials to log in. After successfully logging in, the user uploads a face photo and a full-body photo to the device. In addition, the user inputs the specific situation (e.g., date, work, party) and their preferences (e.g., natural makeup, casual style), which allows the system to obtain the user's specific needs.

[0106] Data transmission process

[0107] The information entered by the user (face photo, full-body photo, situation, preferences) is sent as a single data package from the device to the server. The device notifies the user of the success of the data transmission.

[0108] Data analysis and proposal generation process

[0109] The server analyzes the received data package and inputs it as a prompt to the generative AI model. The generative AI model extracts facial features (shape, skin tone, eyebrow shape, etc.) from the user's face photo and analyzes body shape and style from a full-body photo. It also suggests the optimal makeup, hairstyle, and outfit based on the situation and preferences. The suggestions are sent from the server to the device.

[0110] Prompt Sentence Examples

[0111] "User uploaded a photo for a date. She prefers a casual look. Please suggest the best makeup and hairstyle for her."

[0112] Display of proposal results

[0113] The device analyzes the generated results received from the server and displays the proposals in a format that is easy for the user to understand. The user can review the proposals and, if they like them, proceed to the next step.

[0114] Reservation and payment processing

[0115] If the user likes the suggestions, they can search for nearby beauty salons, esthetic salons, and clothing stores via their device, make a reservation, and make payment. The reservation information and payment data are sent from the device to the server, which relays this information to the appropriate provider. After the server receives a reservation confirmation from the provider, the server processes the successful payment and displays a notification on the device that the reservation and payment have been completed.

[0116] Specific examples

[0117] For example, consider the case where a user uploads a photo of their face and a full body in a situation where they want to know what kind of hairstyle is good for their date. If the user enters "I like casual styles," the system operates as follows:

[0118] 1. User input processing: The user uploads a photo of their face and a full-body photo, and enters the situation and preferences.

[0119] 2. Data transmission process: The device sends a face photo, a full-body photo, the situation, and preference information to the server.

[0120] 3. Data analysis and proposal generation process: The server analyzes the received information and proposes natural curly hair as a "casual hairstyle suitable for a date."

[0121] 4. Displaying the proposal results: The device displays the proposals for natural curly hair to the user.

[0122] 5. Booking and payment processing: The user decides to try the suggested hairstyle, searches for a nearby hair salon, books a reservation and makes payment.

[0123] In this way, users can easily find the makeup, hairstyle, and outfit that suits them, and can smoothly make reservations and make payments. This system can provide the best service, especially to users who are interested in beauty and fashion but are short on time.

[0124] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0125] Specific explanation of processing steps

[0126] Step 1:

[0127] The user launches the application and goes to the login screen. The user enters their account information (username, password) and clicks the login button. The device sends this authentication information as input to the server. The server receives the authentication information and compares it with a database to determine whether the authentication was successful. If the authentication is successful, the server returns the information to the device, and the device outputs the home screen.

[0128] Step 2:

[0129] The user takes or selects a face photo and a full-body photo within the application and clicks the upload button. The device temporarily stores these photo data in memory. Next, a form is displayed in which the user can input the situation (e.g., date, work, party) and preferences (e.g., natural makeup, casual style). The user enters this information and clicks the submit button. These inputs are then compiled into a single data package.

[0130] Step 3:

[0131] The device sends the compiled data package (face photo, full-body photo, situation, and preference information) as input to the server. The server receives this data package and confirms its receipt. After confirmation, the server outputs a notification to the device that reception is complete. The device then notifies the user that the data transmission was successful.

[0132] Step 4:

[0133] The server analyzes the received data package. This analysis process uses facial recognition software (e.g., OpenCV) and data analysis software (e.g., TensorFlow). The server first extracts facial features (e.g., face shape, skin tone, eyebrow shape, etc.) from the face photo, and then analyzes body shape and style from the full-body photo. Next, it generates a prompt based on the situation and preferences. An example prompt is, "Please suggest makeup and hairstyle that suits this face photo and situation."

[0134] Step 5:

[0135] The server inputs the generated prompt into the generative AI model, which then suggests the best makeup, hairstyle, and outfit for the user based on the prompt. The generative AI model returns the generated suggestions (e.g., natural curly hair, beige trench coat, denim pants) to the server, which then receives them and sends them to the device.

[0136] Step 6:

[0137] The device analyzes the generated results received from the server and displays the suggestions in a user-friendly format. This allows the user to see specific makeup, hairstyle, and outfit suggestions. If the user is satisfied with the suggestions, they can proceed to the next step based on the suggestions.

[0138] Step 7:

[0139] If the user wants to try out the suggested service, they use the device's interface to search for nearby beauty salons, esthetic salons, and clothing stores. The device uses the user's current location information as input. The user selects the desired store from the displayed list and sets the reservation date and time. When the user clicks the reservation button, the device sends this reservation information to the server.

[0140] Step 8:

[0141] The server processes the received reservation information and relays it to the appropriate vendor. Once the server receives confirmation of the reservation from the vendor, the server initiates the payment process and notifies the terminal of successful payment. Finally, the terminal displays a notification to the user confirming the reservation and payment.

[0142] This allows users to easily find makeup, hairstyles, and outfits that suit them, and also makes reservations and payments smoothly.

[0143] (Application example 1)

[0144] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0145] In conventional beauty consulting and service booking systems, it was difficult for users to find the best makeup and hairstyle for themselves, and it was also difficult to smoothly book and pay with service providers, which caused many users to feel tedious and time-consuming.

[0146] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0147] In this invention, the server includes means for generating makeup, hairstyles, and outfits based on a user's facial photograph and style photograph using a generative AI model, means for transmitting the generated results to the terminal, and means for searching for and selecting a service provider for which a reservation has been made. This allows the user to easily find the makeup, hairstyle, and outfit that best suits them, and further allows for smooth booking and payment of related services all at once.

[0148] "User" refers to an individual who uses the system to upload a photo of their face and style, and input the situation and preferences.

[0149] "Mouthshot" refers to a photograph used to clearly show a user's facial features.

[0150] A "style photo" refers to a photo that shows the user's entire body and is used to show features such as body shape and clothing.

[0151] "Situation" refers to a specific scene or purpose for the makeup, hairstyle, and coordination desired by the user.

[0152] "Preferences" refer to the style and trends of makeup, hairstyle, and outfits desired by the user.

[0153] "Terminal" refers to an electronic device that a user uses to upload photos, input information, and communicate with a server.

[0154] "Server" refers to the central processing unit that receives and analyzes information sent by the user, generates optimal makeup, hairstyles, and outfits using a generative AI model, and sends the results to the terminal.

[0155] "Analysis" refers to the process by which the server analyzes the face photo, style photo, situation, and preference information it receives and makes appropriate suggestions based on that information.

[0156] A "generative AI model" refers to an artificial intelligence model that uses machine learning algorithms to analyze a user's characteristics and suggest optimal makeup, hairstyles, and outfits.

[0157] "Makeup, hairstyle, and coordination" refers to specific beauty and fashion suggestions that AI suggests based on the user's face photo, style photo, situation, and preferences.

[0158] "Generated results" refers to the makeup, hairstyle, and outfit suggestions generated by the generative AI model.

[0159] "Display" refers to the process by which the terminal visually shows the generated results to the user.

[0160] "Reservation" refers to a user making a reservation with a specific service provider based on the makeup, hairstyle, and outfit suggestions.

[0161] "Payment" refers to the process of paying for the service a user has reserved.

[0162] "Service provider" refers to a business that provides beauty and fashion-related services to users, such as a beauty salon, esthetic salon, or clothing store.

[0163] "Search" refers to the process performed by a terminal to find a particular service provider based on a user request.

[0164] "Selection" refers to the act of a user selecting a particular service provider from the search results.

[0165] System Overview

[0166] This invention is a system that allows users to receive recommendations for the best makeup, hairstyle, and outfits for themselves, and then make reservations and payments with on-site service providers all in one place. The system begins by users uploading photos of their face and style using a terminal, and inputting the situation and preferences. A specific embodiment of this system is described in detail below.

[0167] Program processing flow

[0168] User Input Processing

[0169] When a user launches the application, the device requests the user's account authentication information and prompts the user to log in. After logging in, the user uploads a face photo and a full-body photo, and enters the situation of the day (e.g., date, work, party) and their preferences (e.g., natural makeup, casual style).

[0170] Data transmission process

[0171] The device sends the facial photo, full-body photo, situation, and preference information acquired from the user to the server as a single data package. After the transmission is successful, the device notifies the user of the success.

[0172] Data analysis and proposal generation process

[0173] The server analyzes the received data and inputs it into a generative AI model. The generative AI model extracts facial features (shape, skin tone, eyebrow shape, etc.) from the user's face photo and analyzes the user's body type and style from a full-body photo. It then generates optimal makeup, hairstyles, and outfits based on the situation and preferences. The generated suggestions are sent from the server to the device.

[0174] Display of proposal results

[0175] The device analyzes the generated results received from the server and displays them in a format that is easy for the user to understand. The suggestions include the most suitable makeup, hairstyle, and outfit. The user checks the displayed suggestions and, if they like them, proceeds to the next step.

[0176] Reservation and payment processing

[0177] If the user likes the suggestions, they use the terminal interface to search for relevant service providers (e.g., hair salons, esthetic salons, clothing stores, etc.) and make a reservation and payment. The terminal sends the reservation information and payment data to the server, which relays it to the appropriate provider. The provider's reservation confirmation arrives at the server, which processes the successful payment. Finally, the terminal displays a notification that the reservation and payment have been completed.

[0178] Example of a system

[0179] When applying this system as a smartphone application, the following specific examples can be considered.

[0180] Examples:

[0181] For example, consider the case where a user uploads a photo of their face and a full body in a situation where they want to know what kind of hairstyle is good for their date. If the user enters "I like casual styles," the system operates as follows:

[0182] 1. The user uploads a photo of their face and a full-body photo, and enters the situation and preferences.

[0183] 2. The device sends a photo of the face, a full-body photo, the situation, and preference information to the server.

[0184] 3. The server analyzes the received information and uses a generative AI model to suggest natural curls as a "casual hairstyle suitable for a date."

[0185] 4. The device will display natural curl suggestions to the user.

[0186] 5. The user decides to try the suggested hairstyle and searches for a nearby hair salon, makes a reservation, and pays.

[0187] In this way, users can easily find the makeup, hairstyle, and outfit that suits them, and can also smoothly book and pay for related services.

[0188] Example prompts to input to a generative AI model:

[0189] "Please extract the user's features from face and full-body photos, and suggest the best makeup, hairstyle, and outfit based on the situation (date) and preference (casual)."

[0190] This system makes it possible to provide optimal services to users who are interested in beauty and fashion but don't have much time.

[0191] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0192] Step 1: User Input Processing

[0193] A user launches the application and enters their account authentication information on the login screen. After logging in, they upload a face photo and a full-body photo, and enter the situation of the day (e.g., date, work, party) and their preferences (e.g., natural makeup, casual style). This allows the user's characteristic information to be collected. The input data includes the face photo, full-body photo, situation, and preference information.

[0194] Step 2: Data transmission process

[0195] The device sends the face photo, full-body photo, situation, and preference information acquired from the user to the server as a single data package. The sent data consists of an image file and text information. When the server receives the data, it notifies the device of the success. The device then notifies the user of the success of the data transmission.

[0196] Step 3: Data analysis and proposal generation process

[0197] The server inputs the received data into a generative AI model for analysis. The generative AI model extracts facial features (e.g., facial contours, skin tone, eyebrow shape, etc.) from the face photo and analyzes body shape and clothing characteristics from the full-body photo. Next, it generates optimal makeup, hairstyle, and outfits based on the situation and preferences. The generated suggestions consist of specific advice on makeup, hairstyle, and outfits. Finally, these suggestions are sent from the server to the device.

[0198] Step 4: Displaying the proposed results

[0199] The terminal analyzes the generated results received from the server and displays them in a format that is easy for the user to understand. They are displayed in a specific screen layout (for example, images and descriptions of makeup, hairstyles, and outfits). The user checks these suggestions and, if they like them, proceeds to the next step. The input is the generated results from the server, and the output is the suggestions that the user visually checks.

[0200] Step 5: Booking and payment processing

[0201] If the user likes the proposed service, they use the terminal interface to search for nearby service providers (e.g., hair salons, esthetic salons, clothing stores, etc.). The user selects the desired service provider from the search results and makes a reservation and makes a payment. The terminal sends the reservation information and payment data as a single data package to the server. The server relays the received information and transmits the reservation information to the appropriate service provider. The server also receives reservation confirmation from the provider and notifies the user's terminal. The user can finally confirm that the reservation and payment have been completed. The input is the reservation and payment information, and the output is a notification that the reservation and payment have been completed.

[0202] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0203] System Overview

[0204] This invention is a system that allows users to receive recommendations for the best makeup, hairstyle, and outfits for themselves, and to make reservations and payments at beauty salons, esthetic salons, and clothing stores all in one place. Furthermore, this invention incorporates an emotion engine that analyzes the user's emotions, making it possible to make optimal recommendations based on the user's emotions. This system begins when the user uploads a photo of their face and style, and inputs the situation and preferences. Specific embodiments of this system are described in detail below.

[0205] Program processing flow

[0206] 1. User Input Processing

[0207] When a user launches the application, the device prompts the user for account authentication information, and the user enters the login information. After logging in, the user takes or uploads a face photo and a full-body photo. The user also enters the situation of the day (e.g., date, work, party) and their preferences (e.g., natural makeup, casual style).

[0208] 2. Emotion analysis processing

[0209] The device sends a facial photo to the emotion engine, which then analyzes the user's emotions from the facial photo. The emotion engine recognizes emotions such as happiness, surprise, sadness, and anger, and returns the analysis results to the device. The device then sends the emotion analysis results to the server.

[0210] 3. Data transmission process

[0211] The device sends the facial photo, full-body photo, situation, preference information, and emotion analysis results acquired from the user to the server as a single data package. After the transmission is successful, the device notifies the user of the success.

[0212] 4. Data analysis and proposal generation process

[0213] The server inputs the received data into a generative AI model for analysis. The generative AI model extracts facial features (shape, skin tone, eyebrow shape, etc.) from the user's face photo and analyzes the user's body type and style from a full-body photo. It then generates optimal makeup, hairstyles, and outfits based on the situation, preferences, and emotional analysis results. The generated suggestions are sent from the server to the device.

[0214] 5. Display of proposed results

[0215] The device analyzes the generated results received from the server and displays them in a format that is easy for the user to understand. The suggestions include the most suitable makeup, hairstyle, and outfit. The user reviews the suggestions and, if they like them, proceeds to the next step.

[0216] 6. Booking and payment processing

[0217] If the user likes the suggestions, they use the device's interface to search for nearby beauty salons, esthetic salons, and clothing stores, make a reservation, and pay. The device then sends the reservation information and payment data to the server, which relays it to the appropriate provider. The provider then sends a reservation confirmation to the server, which then processes the successful payment. Finally, the device displays a notification that the reservation and payment have been completed.

[0218] Specific examples

[0219] For example, consider a situation where a user uploads a photo of their face and a full body in a situation where they want to know what kind of hairstyle is best for their date. If the user enters "I like casual styles," the emotion engine analyzes the emotion of "happiness" from the user's face. The system works as follows:

[0220] 1. User input processing: The user uploads a photo of their face and a full-body photo, and enters the situation and preferences.

[0221] 2. Emotion analysis processing: The emotion engine analyzes the facial photo, recognizes the emotion of happiness, and sends the results back to the device.

[0222] 3. Data transmission process: The device sends a face photo, a full-body photo, the situation, preferences, and emotion analysis results to the server.

[0223] 4. Data analysis and proposal generation process: Based on the analysis results, the server proposes natural curly hair as a hairstyle suitable for a date that corresponds to casual and happy emotions.

[0224] 5. Displaying the proposal results: The device displays the proposals for natural curly hair to the user.

[0225] 6. Booking and payment processing: If the user likes the suggested hairstyle, they can search for nearby salons, make a booking and make a payment.

[0226] This series of processes allows users to easily find makeup, hairstyles, and outfits that suit them, and also allows for smooth booking and payment. By combining it with an emotion engine, more personalized suggestions can be made based on the user's emotions.

[0227] The processing flow will be explained below.

[0228] Step 1:

[0229] The user launches the application. The device prompts the user for account authentication information, and the user enters the login information.

[0230] Step 2:

[0231] The user takes or uploads a photo of their face and a full-body photo, and then inputs the situation (e.g., date, work, party) and preferences (e.g., natural makeup, casual style).

[0232] Step 3:

[0233] The device sends a facial photo to the emotion engine, which analyzes the user's facial photo and recognizes emotions such as happiness, surprise, sadness, and anger. The analysis results are then sent back to the device.

[0234] Step 4:

[0235] The terminal transmits the face photo, the style photo, the situation, the preference information, and the emotion analysis result to the server as a single data package. After the transmission is successful, the terminal notifies the user of the success.

[0236] Step 5:

[0237] The server inputs the received data into a generative AI model for analysis. The generative AI model extracts facial features (shape, skin tone, eyebrow shape, etc.) from the user's face photo and analyzes the user's body shape and style from a full-body photo.

[0238] Step 6:

[0239] The server generates the optimal makeup, hairstyle, and outfit based on the analysis results, situation, preferences, and emotion analysis results, and then sends the generated results to the device.

[0240] Step 7:

[0241] The device analyzes the generated results received from the server and displays them in an easy-to-understand format for the user. The user checks the suggestions and, if they like them, proceeds to the next step.

[0242] Step 8:

[0243] If the user likes the suggestions, they can use the device's interface to search for nearby beauty salons, esthetic salons, and clothing stores, and a list of businesses that can provide the suggested services will be displayed on the device.

[0244] Step 9:

[0245] The user selects a vendor and specifies the reservation date and time on a new reservation screen. After the selection and reservation information are confirmed, the terminal sends the reservation information and payment data to the server.

[0246] Step 10:

[0247] The server relays the reservation information to the vendor and receives confirmation of the reservation. The server processes the payment using PayPay and confirms the payment was successful.

[0248] Step 11:

[0249] The server sends a notification of completion of reservation and payment to the terminal. The terminal notifies the user that reservation and payment have been completed.

[0250] This detailed process flow allows users to easily find the makeup, hairstyle, and outfit that suits them, and also allows them to receive personalized suggestions based on emotion analysis. In addition, reservations and payments can be made smoothly, providing optimal service even to busy users.

[0251] Example 2

[0252] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0253] Conventional beauty recommendation systems have the problem that it takes a lot of time and effort for users to find the makeup, hairstyle, and coordination that is best suited to them. Furthermore, since the recommendations do not reflect the user's emotions or situation, the recommendations provided are difficult to personalize. Furthermore, users have to make separate reservations and payments at beauty salons, esthetic salons, and clothing stores based on the recommendations, making the process cumbersome.

[0254] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0255] In this invention, the server includes: means for a user to upload a face photo and a style photo; means for a user to input a situation and preferences; means for a terminal to transmit the face photo, the style photo, the situation, and the preference information to the server; means for the server to input the received information into a generative AI model, analyze facial features and style, and generate optimal makeup, hairstyle, and outfits; means for the terminal to transmit the face photo to an emotion analysis engine and analyze emotions; means for transmitting the emotion analysis results to the server; means for the server to generate a proposal using the emotion analysis results; means for the server to transmit the generation results to the terminal; means for the terminal to display the generation results to the user; means for the user to make a reservation and payment based on the generation results; and means for the server to process the reservation and payment. This allows a user to quickly receive optimal beauty suggestions personalized to them and further allows them to make a reservation and payment based on the proposals all at once.

[0256] "User" refers to any individual or organization that uses this system.

[0257] "Terminal" refers to an electronic device (e.g., a smartphone or PC) operated by a user.

[0258] A "server" refers to a computer that manages the entire system, processes and stores data, and runs generative AI models.

[0259] "Face photo" refers to an image file of a user's face.

[0260] A "style photo" refers to an image file that captures the user's entire body.

[0261] "Situation" refers to the situation or event of the day (e.g., date, work, party) entered by the user.

[0262] "Preferences" refers to personal style and makeup preferences entered by the user (e.g., natural makeup, casual style).

[0263] A "generative AI model" refers to an artificial intelligence model that generates optimal makeup, hairstyles, and outfits based on input data.

[0264] An "emotion analysis engine" refers to software that analyzes emotions from a user's facial photo and provides the results.

[0265] "Data package" refers to a data set that compiles user-entered facial photos, style photos, situations, preference information, and emotion analysis results.

[0266] A "reservation" refers to a user setting a time and date at a store such as a beauty salon, esthetic salon, or clothing store to receive the proposed beauty service.

[0267] "Payment" means the online payment of the service booked by the User.

[0268] MODE FOR CARRYING OUT THE INVENTION

[0269] This invention is a system that allows users to receive recommendations for the best makeup, hairstyle, and outfits for themselves, and to make reservations and payments at hair salons, esthetic salons, and clothing stores all in one place. Furthermore, by incorporating an emotion analysis engine, it is possible to make optimal recommendations based on the user's emotions. This system mainly uses the following hardware and software:

[0270] Hardware and software used

[0271] Device: Electronic device operated by the user, such as a smartphone or computer

[0272] Server: A computer that manages the entire system, processes and stores data, and runs generative AI models.

[0273] Generative AI model: An artificial intelligence model that generates optimal makeup, hairstyles, and outfits based on user information (e.g., "StyleAI")

[0274] Emotion analysis engine: Software that analyzes emotions from a user's facial photo and provides the results (e.g., "EmotionAI")

[0275] Specific examples of programs

[0276] The system starts by having users upload a photo of their face and style, and input the situation and preferences.

[0277] 1. User Input Processing

[0278] When a user launches the application, the device prompts the user for account authentication information, and the user enters the login information. After successful login, the user takes or uploads a face photo and a full-body photo, and enters the situation (e.g., date, work, party) and preferences (e.g., natural makeup, casual style).

[0279] 2. Emotion analysis processing

[0280] The device sends a facial photo to the emotion analysis engine, which then analyzes the user's emotions from the facial photo. The emotion analysis engine recognizes emotions such as happiness, surprise, sadness, and anger, and returns the analysis results to the device. The device then sends the emotion analysis results to the server.

[0281] 3. Data transmission process

[0282] The device sends the facial photo, full-body photo, situation, preference information, and emotion analysis results acquired from the user to the server as a single data package. After the transmission is successful, the device notifies the user of the success.

[0283] 4. Data analysis and proposal generation process

[0284] The server inputs the received data into a generative AI model. The generative AI model extracts facial features (shape, skin tone, eyebrow shape, etc.) from the user's face photo and analyzes the user's body type and style from a full-body photo. It then generates optimal makeup, hairstyles, and outfits based on the situation, preferences, and emotional analysis results. The generated suggestions are sent from the server to the device.

[0285] 5. Display of proposed results

[0286] The device analyzes the generated results received from the server and displays them in a format that is easy for the user to understand. The suggestions include the most suitable makeup, hairstyle, and outfit. The user reviews the suggestions and, if they like them, proceeds to the next step.

[0287] 6. Booking and payment processing

[0288] If the user likes the suggestions, they use the device's interface to search for nearby beauty salons, esthetic salons, and clothing stores, make a reservation, and pay. The device then sends the reservation information and payment data to the server, which relays it to the appropriate provider. The provider then sends a reservation confirmation to the server, which then processes the successful payment. Finally, the device displays a notification that the reservation and payment have been completed.

[0289] Specific examples

[0290] For example, if a user uploads a photo of their face and a full body photo in a situation where they have a date and want to know what kind of hairstyle is nice, the following will happen.

[0291] Users upload photos of their face and body, and enter the situation and preferences.

[0292] The emotion analysis engine analyzes the facial photo, recognizes the emotion of happiness, and sends the results back to the device, which then sends the emotion analysis results to the server.

[0293] The device sends a photo of the face, a full-body photo, the situation, preferences, and emotional analysis results to the server.

[0294] The server inputs the data into a generative AI model, which suggests natural curls as a hairstyle suitable for a date, corresponding to casual and happy feelings.

[0295] The device displays natural curl suggestions to the user.

[0296] The user likes the suggested hairstyle, searches for a nearby hair salon, makes a reservation, and makes payment.

[0297] This series of processes allows users to easily find the makeup, hairstyle, and outfit that suits them, and also allows for smooth booking and payment. By combining it with a sentiment analysis engine, more personalized suggestions can be made based on the user's emotions.

[0298] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0299] Program processing flow

[0300] Step 1: User Input Processing

[0301] Input: User login information, face photo, full-body photo, situation, preferences

[0302] Output: Face photo, full body photo, situation, preference data package

[0303] Specific behavior:

[0304] 1. The user launches the application and enters login information (username and password) into the terminal.

[0305] 2. The device sends the login information to the server for authentication. If the login is successful, the device displays a screen for the user to take or upload a face photo and a full-body photo.

[0306] 3. The user uploads a face photo and a full-body photo. The device then displays a situation input screen, and the user selects an appropriate situation (e.g., date, work, party).

[0307] 4. Next, the device displays a preference input screen, and the user selects their preference (e.g., natural makeup, casual style).

[0308] Step 2: Sentiment Analysis Processing

[0309] Input: Face photo

[0310] Output: Emotion analysis results

[0311] Specific behavior:

[0312] 1. The device sends a photo of the user's face to an emotion analysis engine (e.g., "EmotionAI").

[0313] 2. The emotion analysis engine analyzes the facial photo, recognizes emotions such as happiness, surprise, sadness, and anger, and sends the results back to the device.

[0314] 3. The device receives the emotion analysis results and sends them to the server.

[0315] Step 3: Data transmission process

[0316] Input: Face photo, full-body photo, situation, preferences, emotion analysis results

[0317] Output: Send data package to server

[0318] Specific behavior:

[0319] 1. The device compiles the facial photo, full-body photo, situation, preference information, and emotion analysis results obtained from the user into a single data package.

[0320] 2. The device sends this data package to the server, and after successful transmission, the device notifies the user of the success.

[0321] Step 4: Data analysis and proposal generation process

[0322] Input: Data package (face photo, full-body photo, situation, preferences, emotion analysis results)

[0323] Output: Suggestions (optimal makeup, hairstyle, outfit)

[0324] Specific behavior:

[0325] 1. The server inputs the received data package into a generative AI model (e.g., "StyleAI") for analysis.

[0326] 2. The generative AI model extracts facial features (shape, skin tone, eyebrow shape, etc.) from a photo of the user's face and analyzes the user's body shape and style from a full-body photo.

[0327] 3. The server generates the optimal makeup, hairstyle, and outfit based on the situation, preferences, and emotional analysis results.

[0328] 4. The generated proposal is sent from the server to the device.

[0329] Step 5: Displaying the proposed results

[0330] Input: Suggestion details (best makeup, hairstyle, outfit)

[0331] Output: Suggestion results to be displayed

[0332] Specific behavior:

[0333] 1. The terminal analyzes the generated results received from the server and displays them in a format that is easy for the user to understand.

[0334] 2. The suggestions include the best makeup, hairstyle, and outfit.

[0335] 3. The user reviews the suggestions and, if they like them, proceeds to the next step.

[0336] Step 6: Booking and payment processing

[0337] Input: Proposal details, reservation information, payment information

[0338] Output: Notification of reservation and payment completion

[0339] Specific behavior:

[0340] 1. If the user likes the suggestions, the device provides an interface for searching for nearby hair salons, beauty salons, and clothing stores.

[0341] 2. The user selects the desired store and enters reservation information (date, time, service details, etc.).

[0342] 3. The terminal prompts the user to enter payment information (such as credit card information), after which the terminal sends the reservation information and payment data to the server.

[0343] 4. The server relays the reservation information and payment data to the appropriate vendor, and the vendor sends a confirmation of the reservation to the server.

[0344] 5. The server processes the successful payment and finally displays a notification on the terminal that the reservation and payment are complete.

[0345] (Application example 2)

[0346] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0347] While conventional beauty recommendation systems can suggest optimal makeup, hairstyles, and outfits to users, they face the challenge of making it difficult to actually try out and confirm the suggestions. Furthermore, when making reservations and paying at beauty salons or clothing stores based on the suggestions, multiple steps are required, which is cumbersome for users. This results in limited accuracy of the suggestions and a poor user experience.

[0348] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for generating makeup, hairstyles, and outfits based on the user's face photo, style photo, situation, and preferences using a generative AI model, means for transmitting the generated results to the terminal, and means for the terminal to have the user try on the generated results using a virtual reality device. This allows the user to intuitively check the proposed styles in real time, and also enables reservations and payments to be made all at once.

[0349] "User" refers to an individual who uses this system.

[0350] "Facial photo" refers to image data of the user's face.

[0351] A "style photo" refers to image data that shows the user's entire body.

[0352] "Situation" refers to the purpose or situation in which a user uses the system.

[0353] "Preferences" refer to the fashion and makeup styles desired by the user.

[0354] "Terminal" refers to a device used to use this system, such as a smartphone, tablet, or PC.

[0355] "Server" refers to a remote computer system that communicates with and processes data from a terminal.

[0356] A "generative AI model" refers to an artificial intelligence algorithm that suggests optimal makeup, hairstyles, and outfits based on a user's face photo, style photo, situation, and preferences.

[0357] "Generated results" refers to the makeup, hairstyle, and outfit suggestions obtained using the generative AI model.

[0358] "Means for displaying" refers to the function of visually showing the generated results on the terminal screen.

[0359] "Reservation" refers to the act of a user reserving services in advance at a beauty salon, clothing store, or the like in order to actually perform the suggested makeup or hairstyle.

[0360] "Payment" refers to the act of paying in advance for the service a user uses.

[0361] "Virtual reality device" refers to equipment (e.g., a head-mounted display or smart glasses) that allows users to try out generated results in a virtual space.

[0362] "Trying on" refers to a user virtually experiencing suggested makeup, hairstyles, and outfits using a virtual reality device.

[0363] This invention is a system that allows users to receive personalized recommendations for their best makeup, hairstyle, and outfits, and to make reservations and payments at beauty salons and clothing stores all in one place. The system combines an emotion analysis engine with a generative AI model and uses virtual reality devices to enhance the user experience.

[0364] System Overview

[0365] The server receives the face and style photos uploaded by the user, as well as the situation and preference information, and uses a generative AI model to generate the optimal makeup, hairstyle, and outfit.The generated results are then sent to the device and displayed to the user.

[0366] The terminal sends a photo of the user's face and style to the server in response to user operation, receives the generated results, and displays them. The user can also try out the generated results using a virtual reality device. Furthermore, the user can make reservations and make payments at nearby beauty salons or clothing stores based on the generated results they like.

[0367] Hardware and software used

[0368] Hardware:

[0369] Devices such as smartphones, tablets, and computers

[0370] Virtual reality devices (e.g., head-mounted displays, smart glasses)

[0371] software:

[0372] Emotion Analysis Engine: An algorithm for analyzing emotions from user facial photos

[0373] Generative AI model: An artificial intelligence that generates optimal makeup, hairstyles, and outfits based on information submitted by the user.

[0374] API: Manages the sending and receiving of data between the device and the server

[0375] Data processing and calculation

[0376] The server processes the following data:

[0377] Receives a photo of your face and style, and uses image analysis algorithms to extract facial features (shape, skin tone, eyebrow shape, etc.) and body shape

[0378] The situation (date, work, party, etc.), preferences (natural makeup, casual style, etc.) and emotion analysis results (happiness, surprise, sadness, anger, etc.) are taken into account and input into the generative AI model.

[0379] The generative AI model generates optimal makeup, hairstyle, and outfit suggestions, and sends the results to the device.

[0380] Specific examples

[0381] For example, consider a situation where a user uploads a photo of their face and a full body in a situation where they want to know what kind of hairstyle is best for their date. If the user enters "I like casual styles," and the emotion engine analyzes the emotion "happy" from the user's face, the server will perform the following steps:

[0382] Using a generative AI model based on the situation, preferences, and emotional analysis, the app suggests hairstyles suitable for casual dates.

[0383] The proposed natural curls are sent to the device, which then displays them to the user.

[0384] The device uses a virtual reality device to allow the user to try on suggested hairstyles.

[0385] If the user likes the suggested hairstyle, they can search for nearby hair salons, make a reservation, and pay in one go.

[0386] Prompt Sentence Examples

[0387] An example of a prompt for a generative AI model is:

[0388] "Please suggest the best hairstyle based on the user's face photo, full-body photo, situation (date), preference (casual style), and emotion (happiness)."

[0389] In this way, users can easily find the makeup, hairstyle, and outfit that best suits them, and can check it in real time in the VR space, making reservations and payments smoothly.

[0390] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0391] Step 1:

[0392] The user takes or uploads a photo of their face and whole body, and inputs the situation (e.g., date, work, party) and preferences (e.g., natural makeup, casual style). This information is then added to the device as input data.

[0393] Step 2:

[0394] The device sends a facial photo to an emotion analysis engine, which analyzes the user's emotions (e.g., happiness, surprise, sadness, anger). Specifically, a facial recognition algorithm is used to extract emotions from the image and provide the analysis results.

[0395] Step 3:

[0396] The emotion analysis engine returns the emotion analysis results to the terminal, which then sends these analysis results along with other input data (face photo, full-body photo, situation, preferences, etc.) to the server as a packet.

[0397] Step 4:

[0398] The server analyzes the received data packets and inputs them into a generative AI model. The generative AI model extracts facial features (shape, skin tone, eyebrow shape, etc.) from the user's face photo and analyzes the user's body type and style from a full-body photo. It then generates optimal makeup, hairstyles, and outfits based on the situation, preferences, and emotional analysis results.

[0399] Step 5:

[0400] The server then sends the generated makeup, hairstyle, and outfit suggestions to the device. The generated suggestions are formatted in a way that is intuitively understandable to the user.

[0401] Step 6:

[0402] The device then transmits the generated results to a virtual reality device, which can be a head-mounted display or smart glasses, allowing the user to try out the proposed styles in real time in a virtual space.

[0403] Step 7:

[0404] If the user tries out the suggested makeup, hairstyle, and outfit and likes it, they can search for nearby beauty salons and clothing stores through the device, make a reservation, and pay. The device presents these operations to the user as a series of processes.

[0405] Step 8:

[0406] The terminal sends the user's reservation and payment information to the server, which then relays it to the corresponding vendor. Once the server receives the reservation confirmation from the vendor, the server completes the payment process, sends the final confirmation information to the terminal, and notifies the user.

[0407] This series of processes allows users to easily find the makeup, hairstyle, and outfit that best suits them, and they can then check it in real time in the VR space, making reservations and payments smoothly.

[0408] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0409] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0410] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0411] [Second embodiment]

[0412] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0413] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0414] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0415] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0416] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0417] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0418] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0419] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0420] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0421] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0422] In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0423] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0424] System Overview

[0425] This invention is a system that allows users to receive recommendations for the best makeup, hairstyle, and outfits for themselves, and to make reservations and payments at beauty salons, esthetic salons, and clothing stores all in one place. The system begins when users upload photos of their face and style, and input the situation and preferences. A specific embodiment of this system is described in detail below.

[0426] Program processing flow

[0427] 1. User Input Processing

[0428] When a user launches the application, the device requests the user's account authentication information and prompts the user to log in. After logging in, the user uploads a face photo and a full-body photo. The user also inputs the situation of the day (e.g., date, work, party) and their personal preferences (e.g., natural makeup, casual style).

[0429] 2. Data transmission process

[0430] The device sends the facial photo, full-body photo, situation, and preference information acquired from the user to the server as a single data package. After the transmission is successful, the device notifies the user of the success.

[0431] 3. Data analysis and proposal generation process

[0432] The server analyzes the received data and inputs it into a generative AI model. The generative AI model extracts facial features (shape, skin tone, eyebrow shape, etc.) from the user's face photo and analyzes the user's body type and style from a full-body photo. It then generates optimal makeup, hairstyles, and outfits based on the situation and preferences. The generated suggestions are sent from the server to the device.

[0433] 4. Display of proposed results

[0434] The device analyzes the generated results received from the server and displays them in a format that is easy for the user to understand. The suggestions include the most suitable makeup, hairstyle, and outfit. The user checks the displayed suggestions and, if they like them, proceeds to the next step.

[0435] 5. Booking and payment processing

[0436] If the user likes the suggestions, they use the device's interface to search for nearby beauty salons, esthetic salons, and clothing stores, make a reservation, and pay. The device then sends the reservation information and payment data to the server, which relays it to the appropriate provider. The provider then sends a reservation confirmation to the server, which then processes the successful payment. Finally, the device displays a notification that the reservation and payment have been completed.

[0437] Specific examples

[0438] For example, consider the case where a user uploads a photo of their face and a full body in a situation where they want to know what kind of hairstyle is good for their date. If the user enters "I like casual styles," the system operates as follows:

[0439] 1. User input processing: The user uploads a photo of their face and a full-body photo, and enters the situation and preferences.

[0440] 2. Data transmission process: The device sends a face photo, a full-body photo, the situation, and preference information to the server.

[0441] 3. Data analysis and proposal generation process: The server analyzes the received information and proposes natural curly hair as a "casual hairstyle suitable for a date."

[0442] 4. Displaying the proposal results: The device displays the proposals for natural curly hair to the user.

[0443] 5. Booking and payment processing: The user decides to try the suggested hairstyle, searches for a nearby hair salon, makes a booking and makes payment.

[0444] In this way, users can easily find the makeup, hairstyle, and outfit that suits them, and can smoothly make reservations and make payments. This system provides optimal services to users who are interested in beauty and fashion but don't have much time.

[0445] The processing flow will be explained below.

[0446] Step 1:

[0447] The user launches the application. The device prompts the user for account authentication information, and the user enters the login information.

[0448] Step 2:

[0449] The user takes or uploads a photo of their face and a full-body photo, and then inputs the situation (e.g., date, work, party) and preferences (e.g., natural makeup, casual style).

[0450] Step 3:

[0451] The terminal transmits the face photo, the style photo, the situation, and the preference information to the server as a single data package. After the transmission is successful, the terminal notifies the user of the success.

[0452] Step 4:

[0453] The server inputs the received data into a generative AI model for analysis. The generative AI model extracts facial features (shape, skin tone, eyebrow shape, etc.) from the user's face photo and analyzes the user's body shape and style from a full-body photo.

[0454] Step 5:

[0455] The server generates the optimal makeup, hairstyle, and outfit based on the analysis results and the user's situation and preferences, and sends the generated results to the device.

[0456] Step 6:

[0457] The device analyzes the generated results received from the server and displays them in a format that is easy for the user to understand. The user checks the proposal and, if they like it, proceeds to the next step.

[0458] Step 7:

[0459] The user decides to make a reservation and make payment based on the proposed service. The terminal searches for nearby beauty salons, esthetic salons, and clothing stores and displays a list of businesses that can provide the proposed service.

[0460] Step 8:

[0461] The user selects a vendor and specifies a reservation date and time. After the selection and reservation information are confirmed, the terminal sends the reservation information and payment data to the server.

[0462] Step 9:

[0463] The server relays the reservation information to the merchant and receives confirmation of the reservation. The server processes the payment using PayPay and confirms the payment was successful.

[0464] Step 10:

[0465] The server sends a notification of completion of reservation and payment to the terminal. The terminal notifies the user that reservation and payment have been completed.

[0466] This series of processing flows allows users to easily and quickly receive suggestions for optimal makeup, hairstyles, and outfits, while also smoothly making reservations and making payments.

[0467] Example 1

[0468] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0469] Previously, to receive beauty and fashion recommendations, users had to search for multiple services and stores individually, make reservations, and pay. This was time-consuming and labor-intensive, making it difficult to receive optimal recommendations. Furthermore, technology to automatically generate recommendations tailored to a user's preferences and situation was insufficient, limiting the user experience.

[0470] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0471] In this invention, the server includes a means for inputting prompt text into the generative AI model to suggest makeup, hairstyles, and outfits, a means for the server to transmit the generated results to a terminal, and a means for the user to make reservations and payments based on the generated results. This allows users to receive optimal beauty and fashion suggestions simply by uploading photos of their face and style and inputting the situation and preferences. Furthermore, reservations and payments can be made all at once, significantly reducing the user's time and effort.

[0472] "User" refers to an individual or organization that uses the system.

[0473] "Terminal" refers to the device used by a User (e.g., smartphone, tablet, computer).

[0474] A "face photo" refers to an image of the user's face.

[0475] A "style photo" refers to an image of the user's entire body.

[0476] A "situation" refers to a specific event or purpose (e.g., date, work, party) entered by the user.

[0477] "Preferences" refers to the preferences regarding fashion and makeup input by the user (e.g., natural makeup, casual style).

[0478] "Server" refers to a computer system that processes, stores, and transmits data.

[0479] A "data package" refers to a collection of data including a face photo, style photo, situation, and preference information.

[0480] A "generative AI model" refers to an artificial intelligence algorithm that suggests makeup, hairstyles, and outfits based on input data.

[0481] A "prompt" refers to a specific instruction or request given to a generative AI model.

[0482] "Suggestion generation" refers to the process by which the generative AI model generates the best makeup, hairstyle, and outfit for the user based on the analysis results.

[0483] "Reservation" refers to the act of a user setting a date and time for using services at a beauty salon, esthetic salon, or clothing store based on the content suggested by the user.

[0484] "Payment" refers to the act of paying monetary consideration for the use of a service.

[0485] "Businesses" refer to service providers such as hair salons, beauty salons, and clothing stores.

[0486] "Generated results" refers to the makeup, hairstyle, and outfit suggestions generated by the generative AI model.

[0487] "Relay" refers to the act of the server forwarding information received from the user to the appropriate provider.

[0488] This invention is a system that allows users to receive suggestions for the best makeup, hairstyle, and outfits for themselves, and to make reservations and payments at beauty salons, esthetic salons, and clothing stores all in one place. This system is realized mainly using a terminal, a server, and a generative AI model.

[0489] Specific program description

[0490] User Input Processing

[0491] A user launches the application and uses their device to enter their account credentials to log in. After successfully logging in, the user uploads a face photo and a full-body photo to the device. In addition, the user inputs the specific situation (e.g., date, work, party) and their preferences (e.g., natural makeup, casual style), which allows the system to obtain the user's specific needs.

[0492] Data transmission process

[0493] The information entered by the user (face photo, full-body photo, situation, preferences) is sent as a single data package from the device to the server. The device notifies the user of the success of the data transmission.

[0494] Data analysis and proposal generation process

[0495] The server analyzes the received data package and inputs it as a prompt to the generative AI model. The generative AI model extracts facial features (shape, skin tone, eyebrow shape, etc.) from the user's face photo and analyzes body shape and style from a full-body photo. It also suggests the optimal makeup, hairstyle, and outfit based on the situation and preferences. The suggestions are sent from the server to the device.

[0496] Prompt Sentence Examples

[0497] "User uploaded a photo for a date. She prefers a casual look. Please suggest the best makeup and hairstyle for her."

[0498] Display of proposal results

[0499] The device analyzes the generated results received from the server and displays the proposals in a format that is easy for the user to understand. The user can review the proposals and, if they like them, proceed to the next step.

[0500] Reservation and payment processing

[0501] If the user likes the suggestions, they can search for nearby beauty salons, esthetic salons, and clothing stores via their device, make a reservation, and make payment. The reservation information and payment data are sent from the device to the server, which relays this information to the appropriate provider. After the server receives a reservation confirmation from the provider, the server processes the successful payment and displays a notification on the device that the reservation and payment have been completed.

[0502] Specific examples

[0503] For example, consider the case where a user uploads a photo of their face and a full body in a situation where they want to know what kind of hairstyle is good for their date. If the user enters "I like casual styles," the system operates as follows:

[0504] 1. User input processing: The user uploads a photo of their face and a full-body photo, and enters the situation and preferences.

[0505] 2. Data transmission process: The device sends a face photo, a full-body photo, the situation, and preference information to the server.

[0506] 3. Data analysis and proposal generation process: The server analyzes the received information and proposes natural curly hair as a "casual hairstyle suitable for a date."

[0507] 4. Displaying the proposal results: The device displays the proposals for natural curly hair to the user.

[0508] 5. Booking and payment processing: The user decides to try the suggested hairstyle, searches for a nearby hair salon, books a reservation and makes payment.

[0509] In this way, users can easily find the makeup, hairstyle, and outfit that suits them, and can smoothly make reservations and make payments. This system can provide the best service, especially to users who are interested in beauty and fashion but are short on time.

[0510] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0511] Specific explanation of processing steps

[0512] Step 1:

[0513] The user launches the application and goes to the login screen. The user enters their account information (username, password) and clicks the login button. The device sends this authentication information as input to the server. The server receives the authentication information and compares it with a database to determine whether the authentication was successful. If the authentication is successful, the server returns the information to the device, and the device outputs the home screen.

[0514] Step 2:

[0515] The user takes or selects a face photo and a full-body photo within the application and clicks the upload button. The device temporarily stores these photo data in memory. Next, a form is displayed in which the user can input the situation (e.g., date, work, party) and preferences (e.g., natural makeup, casual style). The user enters this information and clicks the submit button. These inputs are then compiled into a single data package.

[0516] Step 3:

[0517] The device sends the compiled data package (face photo, full-body photo, situation, and preference information) as input to the server. The server receives this data package and confirms its receipt. After confirmation, the server outputs a notification to the device that reception is complete. The device then notifies the user that the data transmission was successful.

[0518] Step 4:

[0519] The server analyzes the received data package. This analysis process uses facial recognition software (e.g., OpenCV) and data analysis software (e.g., TensorFlow). The server first extracts facial features (e.g., face shape, skin tone, eyebrow shape, etc.) from the face photo, and then analyzes body shape and style from the full-body photo. Next, it generates a prompt based on the situation and preferences. An example prompt is, "Please suggest makeup and hairstyle that suits this face photo and situation."

[0520] Step 5:

[0521] The server inputs the generated prompt into the generative AI model, which then suggests the best makeup, hairstyle, and outfit for the user based on the prompt. The generative AI model returns the generated suggestions (e.g., natural curly hair, beige trench coat, denim pants) to the server, which then receives them and sends them to the device.

[0522] Step 6:

[0523] The device analyzes the generated results received from the server and displays the suggestions in a user-friendly format. This allows the user to see specific makeup, hairstyle, and outfit suggestions. If the user is satisfied with the suggestions, they can proceed to the next step based on the suggestions.

[0524] Step 7:

[0525] If the user wants to try out the suggested service, they use the device's interface to search for nearby beauty salons, esthetic salons, and clothing stores. The device uses the user's current location information as input. The user selects the desired store from the displayed list and sets the reservation date and time. When the user clicks the reservation button, the device sends this reservation information to the server.

[0526] Step 8:

[0527] The server processes the received reservation information and relays it to the appropriate vendor. Once the server receives confirmation of the reservation from the vendor, the server initiates the payment process and notifies the terminal of successful payment. Finally, the terminal displays a notification to the user confirming the reservation and payment.

[0528] This allows users to easily find makeup, hairstyles, and outfits that suit them, and also makes reservations and payments smoothly.

[0529] (Application example 1)

[0530] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0531] In conventional beauty consulting and service booking systems, it was difficult for users to find the best makeup and hairstyle for themselves, and it was also difficult to smoothly book and pay with service providers, which caused many users to feel tedious and time-consuming.

[0532] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0533] In this invention, the server includes means for generating makeup, hairstyles, and outfits based on a user's facial photograph and style photograph using a generative AI model, means for transmitting the generated results to the terminal, and means for searching for and selecting a service provider for which a reservation has been made. This allows the user to easily find the makeup, hairstyle, and outfit that best suits them, and further allows for smooth booking and payment of related services all at once.

[0534] "User" refers to an individual who uses the system to upload a photo of their face and style, and input the situation and preferences.

[0535] "Mouthshot" refers to a photograph used to clearly show a user's facial features.

[0536] A "style photo" refers to a photo that shows the user's entire body and is used to show features such as body shape and clothing.

[0537] "Situation" refers to a specific scene or purpose for the makeup, hairstyle, and coordination desired by the user.

[0538] "Preferences" refer to the style and trends of makeup, hairstyle, and outfits desired by the user.

[0539] "Terminal" refers to an electronic device that a user uses to upload photos, input information, and communicate with a server.

[0540] "Server" refers to the central processing unit that receives and analyzes information sent by the user, generates optimal makeup, hairstyles, and outfits using a generative AI model, and sends the results to the terminal.

[0541] "Analysis" refers to the process by which the server analyzes the face photo, style photo, situation, and preference information it receives and makes appropriate suggestions based on that information.

[0542] A "generative AI model" refers to an artificial intelligence model that uses machine learning algorithms to analyze a user's characteristics and suggest optimal makeup, hairstyles, and outfits.

[0543] "Makeup, hairstyle, and coordination" refers to specific beauty and fashion suggestions that AI suggests based on the user's face photo, style photo, situation, and preferences.

[0544] "Generated results" refers to the makeup, hairstyle, and outfit suggestions generated by the generative AI model.

[0545] "Display" refers to the process by which the terminal visually shows the generated results to the user.

[0546] "Reservation" refers to a user making a reservation with a specific service provider based on the makeup, hairstyle, and outfit suggestions.

[0547] "Payment" refers to the process of paying for the service a user has reserved.

[0548] "Service provider" refers to a business that provides beauty and fashion-related services to users, such as a beauty salon, esthetic salon, or clothing store.

[0549] "Search" refers to the process performed by a terminal to find a particular service provider based on a user request.

[0550] "Selection" refers to the act of a user selecting a particular service provider from the search results.

[0551] System Overview

[0552] This invention is a system that allows users to receive recommendations for the best makeup, hairstyle, and outfits for themselves, and then make reservations and payments with on-site service providers all in one place. The system begins by users uploading photos of their face and style using a terminal, and inputting the situation and preferences. A specific embodiment of this system is described in detail below.

[0553] Program processing flow

[0554] User Input Processing

[0555] When a user launches the application, the device requests the user's account authentication information and prompts the user to log in. After logging in, the user uploads a face photo and a full-body photo, and enters the situation of the day (e.g., date, work, party) and their preferences (e.g., natural makeup, casual style).

[0556] Data transmission process

[0557] The device sends the facial photo, full-body photo, situation, and preference information acquired from the user to the server as a single data package. After the transmission is successful, the device notifies the user of the success.

[0558] Data analysis and proposal generation process

[0559] The server analyzes the received data and inputs it into a generative AI model. The generative AI model extracts facial features (shape, skin tone, eyebrow shape, etc.) from the user's face photo and analyzes the user's body type and style from a full-body photo. It then generates optimal makeup, hairstyles, and outfits based on the situation and preferences. The generated suggestions are sent from the server to the device.

[0560] Display of proposal results

[0561] The device analyzes the generated results received from the server and displays them in a format that is easy for the user to understand. The suggestions include the most suitable makeup, hairstyle, and outfit. The user checks the displayed suggestions and, if they like them, proceeds to the next step.

[0562] Reservation and payment processing

[0563] If the user likes the suggestions, they use the terminal interface to search for relevant service providers (e.g., hair salons, esthetic salons, clothing stores, etc.) and make a reservation and payment. The terminal sends the reservation information and payment data to the server, which relays it to the appropriate provider. The provider's reservation confirmation arrives at the server, which processes the successful payment. Finally, the terminal displays a notification that the reservation and payment have been completed.

[0564] Example of a system

[0565] When applying this system as a smartphone application, the following specific examples can be considered.

[0566] Examples:

[0567] For example, consider the case where a user uploads a photo of their face and a full body in a situation where they want to know what kind of hairstyle is good for their date. If the user enters "I like casual styles," the system operates as follows:

[0568] 1. The user uploads a photo of their face and a full-body photo, and enters the situation and preferences.

[0569] 2. The device sends a photo of the face, a full-body photo, the situation, and preference information to the server.

[0570] 3. The server analyzes the received information and uses a generative AI model to suggest natural curls as a "casual hairstyle suitable for a date."

[0571] 4. The device will display natural curl suggestions to the user.

[0572] 5. The user decides to try the suggested hairstyle and searches for a nearby hair salon, makes a reservation, and pays.

[0573] In this way, users can easily find the makeup, hairstyle, and outfit that suits them, and can also smoothly book and pay for related services.

[0574] Example prompts to input to a generative AI model:

[0575] "Please extract the user's features from face and full-body photos, and suggest the best makeup, hairstyle, and outfit based on the situation (date) and preference (casual)."

[0576] This system makes it possible to provide optimal services to users who are interested in beauty and fashion but don't have much time.

[0577] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0578] Step 1: User Input Processing

[0579] A user launches the application and enters their account authentication information on the login screen. After logging in, they upload a face photo and a full-body photo, and enter the situation of the day (e.g., date, work, party) and their preferences (e.g., natural makeup, casual style). This allows the user's characteristic information to be collected. The input data includes the face photo, full-body photo, situation, and preference information.

[0580] Step 2: Data transmission process

[0581] The device sends the face photo, full-body photo, situation, and preference information acquired from the user to the server as a single data package. The sent data consists of an image file and text information. When the server receives the data, it notifies the device of the success. The device then notifies the user of the success of the data transmission.

[0582] Step 3: Data analysis and proposal generation process

[0583] The server inputs the received data into a generative AI model for analysis. The generative AI model extracts facial features (e.g., facial contours, skin tone, eyebrow shape, etc.) from the face photo and analyzes body shape and clothing characteristics from the full-body photo. Next, it generates optimal makeup, hairstyle, and outfits based on the situation and preferences. The generated suggestions consist of specific advice on makeup, hairstyle, and outfits. Finally, these suggestions are sent from the server to the device.

[0584] Step 4: Displaying the proposed results

[0585] The terminal analyzes the generated results received from the server and displays them in a format that is easy for the user to understand. They are displayed in a specific screen layout (for example, images and descriptions of makeup, hairstyles, and outfits). The user checks these suggestions and, if they like them, proceeds to the next step. The input is the generated results from the server, and the output is the suggestions that the user visually checks.

[0586] Step 5: Booking and payment processing

[0587] If the user likes the proposed service, they use the terminal interface to search for nearby service providers (e.g., hair salons, esthetic salons, clothing stores, etc.). The user selects the desired service provider from the search results and makes a reservation and makes a payment. The terminal sends the reservation information and payment data as a single data package to the server. The server relays the received information and transmits the reservation information to the appropriate service provider. The server also receives reservation confirmation from the provider and notifies the user's terminal. The user can finally confirm that the reservation and payment have been completed. The input is the reservation and payment information, and the output is a notification that the reservation and payment have been completed.

[0588] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0589] System Overview

[0590] This invention is a system that allows users to receive recommendations for the best makeup, hairstyle, and outfits for themselves, and to make reservations and payments at beauty salons, esthetic salons, and clothing stores all in one place. Furthermore, this invention incorporates an emotion engine that analyzes the user's emotions, making it possible to make optimal recommendations based on the user's emotions. This system begins when the user uploads a photo of their face and style, and inputs the situation and preferences. Specific embodiments of this system are described in detail below.

[0591] Program processing flow

[0592] 1. User Input Processing

[0593] When a user launches the application, the device prompts the user for account authentication information, and the user enters the login information. After logging in, the user takes or uploads a face photo and a full-body photo. The user also enters the situation of the day (e.g., date, work, party) and their preferences (e.g., natural makeup, casual style).

[0594] 2. Emotion analysis processing

[0595] The device sends a facial photo to the emotion engine, which then analyzes the user's emotions from the facial photo. The emotion engine recognizes emotions such as happiness, surprise, sadness, and anger, and returns the analysis results to the device. The device then sends the emotion analysis results to the server.

[0596] 3. Data transmission process

[0597] The device sends the facial photo, full-body photo, situation, preference information, and emotion analysis results acquired from the user to the server as a single data package. After the transmission is successful, the device notifies the user of the success.

[0598] 4. Data analysis and proposal generation process

[0599] The server inputs the received data into a generative AI model for analysis. The generative AI model extracts facial features (shape, skin tone, eyebrow shape, etc.) from the user's face photo and analyzes the user's body type and style from a full-body photo. It then generates optimal makeup, hairstyles, and outfits based on the situation, preferences, and emotional analysis results. The generated suggestions are sent from the server to the device.

[0600] 5. Display of proposed results

[0601] The device analyzes the generated results received from the server and displays them in a format that is easy for the user to understand. The suggestions include the most suitable makeup, hairstyle, and outfit. The user reviews the suggestions and, if they like them, proceeds to the next step.

[0602] 6. Booking and payment processing

[0603] If the user likes the suggestions, they use the device's interface to search for nearby beauty salons, esthetic salons, and clothing stores, make a reservation, and pay. The device then sends the reservation information and payment data to the server, which relays it to the appropriate provider. The provider then sends a reservation confirmation to the server, which then processes the successful payment. Finally, the device displays a notification that the reservation and payment have been completed.

[0604] Specific examples

[0605] For example, consider a situation where a user uploads a photo of their face and a full body in a situation where they want to know what kind of hairstyle is best for their date. If the user enters "I like casual styles," the emotion engine analyzes the emotion of "happiness" from the user's face. The system works as follows:

[0606] 1. User input processing: The user uploads a photo of their face and a full-body photo, and enters the situation and preferences.

[0607] 2. Emotion analysis processing: The emotion engine analyzes the facial photo, recognizes the emotion of happiness, and sends the results back to the device.

[0608] 3. Data transmission process: The device sends a face photo, a full-body photo, the situation, preferences, and emotion analysis results to the server.

[0609] 4. Data analysis and proposal generation process: Based on the analysis results, the server proposes natural curly hair as a hairstyle suitable for a date that corresponds to casual and happy emotions.

[0610] 5. Displaying the proposal results: The device displays the proposals for natural curly hair to the user.

[0611] 6. Booking and payment processing: If the user likes the suggested hairstyle, they can search for nearby salons, make a booking and make a payment.

[0612] This series of processes allows users to easily find makeup, hairstyles, and outfits that suit them, and also allows for smooth booking and payment. By combining it with an emotion engine, more personalized suggestions can be made based on the user's emotions.

[0613] The processing flow will be explained below.

[0614] Step 1:

[0615] The user launches the application. The device prompts the user for account authentication information, and the user enters the login information.

[0616] Step 2:

[0617] The user takes or uploads a photo of their face and a full-body photo, and then inputs the situation (e.g., date, work, party) and preferences (e.g., natural makeup, casual style).

[0618] Step 3:

[0619] The device sends a facial photo to the emotion engine, which analyzes the user's facial photo and recognizes emotions such as happiness, surprise, sadness, and anger. The analysis results are then sent back to the device.

[0620] Step 4:

[0621] The terminal transmits the face photo, the style photo, the situation, the preference information, and the emotion analysis result to the server as a single data package. After the transmission is successful, the terminal notifies the user of the success.

[0622] Step 5:

[0623] The server inputs the received data into a generative AI model for analysis. The generative AI model extracts facial features (shape, skin tone, eyebrow shape, etc.) from the user's face photo and analyzes the user's body shape and style from a full-body photo.

[0624] Step 6:

[0625] The server generates the optimal makeup, hairstyle, and outfit based on the analysis results, situation, preferences, and emotion analysis results, and then sends the generated results to the device.

[0626] Step 7:

[0627] The device analyzes the generated results received from the server and displays them in an easy-to-understand format for the user. The user checks the suggestions and, if they like them, proceeds to the next step.

[0628] Step 8:

[0629] If the user likes the suggestions, they can use the device's interface to search for nearby beauty salons, esthetic salons, and clothing stores, and a list of businesses that can provide the suggested services will be displayed on the device.

[0630] Step 9:

[0631] The user selects a vendor and specifies the reservation date and time on a new reservation screen. After the selection and reservation information are confirmed, the terminal sends the reservation information and payment data to the server.

[0632] Step 10:

[0633] The server relays the reservation information to the vendor and receives confirmation of the reservation. The server processes the payment using PayPay and confirms the payment was successful.

[0634] Step 11:

[0635] The server sends a notification of completion of reservation and payment to the terminal. The terminal notifies the user that reservation and payment have been completed.

[0636] This detailed process flow allows users to easily find the makeup, hairstyle, and outfit that suits them, and also allows them to receive personalized suggestions based on emotion analysis. In addition, reservations and payments can be made smoothly, providing optimal service even to busy users.

[0637] Example 2

[0638] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0639] Conventional beauty recommendation systems have the problem that it takes a lot of time and effort for users to find the makeup, hairstyle, and coordination that is best suited to them. Furthermore, since the recommendations do not reflect the user's emotions or situation, the recommendations provided are difficult to personalize. Furthermore, users have to make separate reservations and payments at beauty salons, esthetic salons, and clothing stores based on the recommendations, making the process cumbersome.

[0640] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0641] In this invention, the server includes: means for a user to upload a face photo and a style photo; means for a user to input a situation and preferences; means for a terminal to transmit the face photo, the style photo, the situation, and the preference information to the server; means for the server to input the received information into a generative AI model, analyze facial features and style, and generate optimal makeup, hairstyle, and outfits; means for the terminal to transmit the face photo to an emotion analysis engine and analyze emotions; means for transmitting the emotion analysis results to the server; means for the server to generate a proposal using the emotion analysis results; means for the server to transmit the generation results to the terminal; means for the terminal to display the generation results to the user; means for the user to make a reservation and payment based on the generation results; and means for the server to process the reservation and payment. This allows a user to quickly receive optimal beauty suggestions personalized to them and further allows them to make a reservation and payment based on the proposals all at once.

[0642] "User" refers to any individual or organization that uses this system.

[0643] "Terminal" refers to an electronic device (e.g., a smartphone or PC) operated by a user.

[0644] A "server" refers to a computer that manages the entire system, processes and stores data, and runs generative AI models.

[0645] "Face photo" refers to an image file of a user's face.

[0646] A "style photo" refers to an image file that captures the user's entire body.

[0647] "Situation" refers to the situation or event of the day (e.g., date, work, party) entered by the user.

[0648] "Preferences" refers to personal style and makeup preferences entered by the user (e.g., natural makeup, casual style).

[0649] A "generative AI model" refers to an artificial intelligence model that generates optimal makeup, hairstyles, and outfits based on input data.

[0650] An "emotion analysis engine" refers to software that analyzes emotions from a user's facial photo and provides the results.

[0651] "Data package" refers to a data set that compiles user-entered facial photos, style photos, situations, preference information, and emotion analysis results.

[0652] A "reservation" refers to a user setting a time and date at a store such as a beauty salon, esthetic salon, or clothing store to receive the proposed beauty service.

[0653] "Payment" means the online payment of the service booked by the User.

[0654] MODE FOR CARRYING OUT THE INVENTION

[0655] This invention is a system that allows users to receive recommendations for the best makeup, hairstyle, and outfits for themselves, and to make reservations and payments at hair salons, esthetic salons, and clothing stores all in one place. Furthermore, by incorporating an emotion analysis engine, it is possible to make optimal recommendations based on the user's emotions. This system mainly uses the following hardware and software:

[0656] Hardware and software used

[0657] Device: Electronic device operated by the user, such as a smartphone or computer

[0658] Server: A computer that manages the entire system, processes and stores data, and runs generative AI models.

[0659] Generative AI model: An artificial intelligence model that generates optimal makeup, hairstyles, and outfits based on user information (e.g., "StyleAI")

[0660] Emotion analysis engine: Software that analyzes emotions from a user's facial photo and provides the results (e.g., "EmotionAI")

[0661] Specific examples of programs

[0662] The system starts by having users upload a photo of their face and style, and input the situation and preferences.

[0663] 1. User Input Processing

[0664] When a user launches the application, the device prompts the user for account authentication information, and the user enters the login information. After successful login, the user takes or uploads a face photo and a full-body photo, and enters the situation (e.g., date, work, party) and preferences (e.g., natural makeup, casual style).

[0665] 2. Emotion analysis processing

[0666] The device sends a facial photo to the emotion analysis engine, which then analyzes the user's emotions from the facial photo. The emotion analysis engine recognizes emotions such as happiness, surprise, sadness, and anger, and returns the analysis results to the device. The device then sends the emotion analysis results to the server.

[0667] 3. Data transmission process

[0668] The device sends the facial photo, full-body photo, situation, preference information, and emotion analysis results acquired from the user to the server as a single data package. After the transmission is successful, the device notifies the user of the success.

[0669] 4. Data analysis and proposal generation process

[0670] The server inputs the received data into a generative AI model. The generative AI model extracts facial features (shape, skin tone, eyebrow shape, etc.) from the user's face photo and analyzes the user's body type and style from a full-body photo. It then generates optimal makeup, hairstyles, and outfits based on the situation, preferences, and emotional analysis results. The generated suggestions are sent from the server to the device.

[0671] 5. Display of proposed results

[0672] The device analyzes the generated results received from the server and displays them in a format that is easy for the user to understand. The suggestions include the most suitable makeup, hairstyle, and outfit. The user reviews the suggestions and, if they like them, proceeds to the next step.

[0673] 6. Booking and payment processing

[0674] If the user likes the suggestions, they use the device's interface to search for nearby beauty salons, esthetic salons, and clothing stores, make a reservation, and pay. The device then sends the reservation information and payment data to the server, which relays it to the appropriate provider. The provider then sends a reservation confirmation to the server, which then processes the successful payment. Finally, the device displays a notification that the reservation and payment have been completed.

[0675] Specific examples

[0676] For example, if a user uploads a photo of their face and a full body photo in a situation where they have a date and want to know what kind of hairstyle is nice, the following will happen.

[0677] Users upload photos of their face and body, and enter the situation and preferences.

[0678] The emotion analysis engine analyzes the facial photo, recognizes the emotion of happiness, and sends the results back to the device, which then sends the emotion analysis results to the server.

[0679] The device sends a photo of the face, a full-body photo, the situation, preferences, and emotional analysis results to the server.

[0680] The server inputs the data into a generative AI model, which suggests natural curls as a hairstyle suitable for a date, corresponding to casual and happy feelings.

[0681] The device displays natural curl suggestions to the user.

[0682] The user likes the suggested hairstyle, searches for a nearby hair salon, makes a reservation, and makes payment.

[0683] This series of processes allows users to easily find the makeup, hairstyle, and outfit that suits them, and also allows for smooth booking and payment. By combining it with a sentiment analysis engine, more personalized suggestions can be made based on the user's emotions.

[0684] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0685] Program processing flow

[0686] Step 1: User Input Processing

[0687] Input: User login information, face photo, full-body photo, situation, preferences

[0688] Output: Face photo, full body photo, situation, preference data package

[0689] Specific behavior:

[0690] 1. The user launches the application and enters login information (username and password) into the terminal.

[0691] 2. The device sends the login information to the server for authentication. If the login is successful, the device displays a screen for the user to take or upload a face photo and a full-body photo.

[0692] 3. The user uploads a face photo and a full-body photo. The device then displays a situation input screen, and the user selects an appropriate situation (e.g., date, work, party).

[0693] 4. Next, the device displays a preference input screen, and the user selects their preference (e.g., natural makeup, casual style).

[0694] Step 2: Sentiment Analysis Processing

[0695] Input: Face photo

[0696] Output: Emotion analysis results

[0697] Specific behavior:

[0698] 1. The device sends a photo of the user's face to an emotion analysis engine (e.g., "EmotionAI").

[0699] 2. The emotion analysis engine analyzes the facial photo, recognizes emotions such as happiness, surprise, sadness, and anger, and sends the results back to the device.

[0700] 3. The device receives the emotion analysis results and sends them to the server.

[0701] Step 3: Data transmission process

[0702] Input: Face photo, full-body photo, situation, preferences, emotion analysis results

[0703] Output: Send data package to server

[0704] Specific behavior:

[0705] 1. The device compiles the facial photo, full-body photo, situation, preference information, and emotion analysis results obtained from the user into a single data package.

[0706] 2. The device sends this data package to the server, and after successful transmission, the device notifies the user of the success.

[0707] Step 4: Data analysis and proposal generation process

[0708] Input: Data package (face photo, full-body photo, situation, preferences, emotion analysis results)

[0709] Output: Suggestions (optimal makeup, hairstyle, outfit)

[0710] Specific behavior:

[0711] 1. The server inputs the received data package into a generative AI model (e.g., "StyleAI") for analysis.

[0712] 2. The generative AI model extracts facial features (shape, skin tone, eyebrow shape, etc.) from a photo of the user's face and analyzes the user's body shape and style from a full-body photo.

[0713] 3. The server generates the optimal makeup, hairstyle, and outfit based on the situation, preferences, and emotional analysis results.

[0714] 4. The generated proposal is sent from the server to the device.

[0715] Step 5: Displaying the proposed results

[0716] Input: Suggestion details (best makeup, hairstyle, outfit)

[0717] Output: Suggestion results to be displayed

[0718] Specific behavior:

[0719] 1. The terminal analyzes the generated results received from the server and displays them in a format that is easy for the user to understand.

[0720] 2. The suggestions include the best makeup, hairstyle, and outfit.

[0721] 3. The user reviews the suggestions and, if they like them, proceeds to the next step.

[0722] Step 6: Booking and payment processing

[0723] Input: Proposal details, reservation information, payment information

[0724] Output: Notification of reservation and payment completion

[0725] Specific behavior:

[0726] 1. If the user likes the suggestions, the device provides an interface for searching for nearby hair salons, beauty salons, and clothing stores.

[0727] 2. The user selects the desired store and enters reservation information (date, time, service details, etc.).

[0728] 3. The terminal prompts the user to enter payment information (such as credit card information), after which the terminal sends the reservation information and payment data to the server.

[0729] 4. The server relays the reservation information and payment data to the appropriate vendor, and the vendor sends a confirmation of the reservation to the server.

[0730] 5. The server processes the successful payment and finally displays a notification on the terminal that the reservation and payment are complete.

[0731] (Application example 2)

[0732] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0733] While conventional beauty recommendation systems can suggest optimal makeup, hairstyles, and outfits to users, they face the challenge of making it difficult to actually try out and confirm the suggestions. Furthermore, when making reservations and paying at beauty salons or clothing stores based on the suggestions, multiple steps are required, which is cumbersome for users. This results in limited accuracy of the suggestions and a poor user experience.

[0734] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for generating makeup, hairstyles, and outfits based on the user's face photo, style photo, situation, and preferences using a generative AI model, means for transmitting the generated results to the terminal, and means for the terminal to have the user try on the generated results using a virtual reality device. This allows the user to intuitively check the proposed styles in real time, and also enables reservations and payments to be made all at once.

[0735] "User" refers to an individual who uses this system.

[0736] "Facial photo" refers to image data of the user's face.

[0737] A "style photo" refers to image data that shows the user's entire body.

[0738] "Situation" refers to the purpose or situation in which a user uses the system.

[0739] "Preferences" refer to the fashion and makeup styles desired by the user.

[0740] "Terminal" refers to a device used to use this system, such as a smartphone, tablet, or PC.

[0741] "Server" refers to a remote computer system that communicates with and processes data from a terminal.

[0742] A "generative AI model" refers to an artificial intelligence algorithm that suggests optimal makeup, hairstyles, and outfits based on a user's face photo, style photo, situation, and preferences.

[0743] "Generated results" refers to the makeup, hairstyle, and outfit suggestions obtained using the generative AI model.

[0744] "Means for displaying" refers to the function of visually showing the generated results on the terminal screen.

[0745] "Reservation" refers to the act of a user reserving services in advance at a beauty salon, clothing store, or the like in order to actually perform the suggested makeup or hairstyle.

[0746] "Payment" refers to the act of paying in advance for the service a user uses.

[0747] "Virtual reality device" refers to equipment (e.g., a head-mounted display or smart glasses) that allows users to try out generated results in a virtual space.

[0748] "Trying on" refers to a user virtually experiencing suggested makeup, hairstyles, and outfits using a virtual reality device.

[0749] This invention is a system that allows users to receive personalized recommendations for their best makeup, hairstyle, and outfits, and to make reservations and payments at beauty salons and clothing stores all in one place. The system combines an emotion analysis engine with a generative AI model and uses virtual reality devices to enhance the user experience.

[0750] System Overview

[0751] The server receives the face and style photos uploaded by the user, as well as the situation and preference information, and uses a generative AI model to generate the optimal makeup, hairstyle, and outfit.The generated results are then sent to the device and displayed to the user.

[0752] The terminal sends a photo of the user's face and style to the server in response to user operation, receives the generated results, and displays them. The user can also try out the generated results using a virtual reality device. Furthermore, the user can make reservations and make payments at nearby beauty salons or clothing stores based on the generated results they like.

[0753] Hardware and software used

[0754] Hardware:

[0755] Devices such as smartphones, tablets, and computers

[0756] Virtual reality devices (e.g., head-mounted displays, smart glasses)

[0757] software:

[0758] Emotion Analysis Engine: An algorithm for analyzing emotions from user facial photos

[0759] Generative AI model: An artificial intelligence that generates optimal makeup, hairstyles, and outfits based on information submitted by the user.

[0760] API: Manages the sending and receiving of data between the device and the server

[0761] Data processing and calculation

[0762] The server processes the following data:

[0763] Receives a photo of your face and style, and uses image analysis algorithms to extract facial features (shape, skin tone, eyebrow shape, etc.) and body shape

[0764] The situation (date, work, party, etc.), preferences (natural makeup, casual style, etc.) and emotion analysis results (happiness, surprise, sadness, anger, etc.) are taken into account and input into the generative AI model.

[0765] The generative AI model generates optimal makeup, hairstyle, and outfit suggestions, and sends the results to the device.

[0766] Specific examples

[0767] For example, consider a situation where a user uploads a photo of their face and a full body in a situation where they want to know what kind of hairstyle is best for their date. If the user enters "I like casual styles," and the emotion engine analyzes the emotion "happy" from the user's face, the server will perform the following steps:

[0768] Using a generative AI model based on the situation, preferences, and emotional analysis, the app suggests hairstyles suitable for casual dates.

[0769] The proposed natural curls are sent to the device, which then displays them to the user.

[0770] The device uses a virtual reality device to allow the user to try on suggested hairstyles.

[0771] If the user likes the suggested hairstyle, they can search for nearby hair salons, make a reservation, and pay in one go.

[0772] Prompt Sentence Examples

[0773] An example of a prompt for a generative AI model is:

[0774] "Please suggest the best hairstyle based on the user's face photo, full-body photo, situation (date), preference (casual style), and emotion (happiness)."

[0775] In this way, users can easily find the makeup, hairstyle, and outfit that best suits them, and can check it in real time in the VR space, making reservations and payments smoothly.

[0776] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0777] Step 1:

[0778] The user takes or uploads a photo of their face and whole body, and inputs the situation (e.g., date, work, party) and preferences (e.g., natural makeup, casual style). This information is then added to the device as input data.

[0779] Step 2:

[0780] The device sends a facial photo to an emotion analysis engine, which analyzes the user's emotions (e.g., happiness, surprise, sadness, anger). Specifically, a facial recognition algorithm is used to extract emotions from the image and provide the analysis results.

[0781] Step 3:

[0782] The emotion analysis engine returns the emotion analysis results to the terminal, which then sends these analysis results along with other input data (face photo, full-body photo, situation, preferences, etc.) to the server as a packet.

[0783] Step 4:

[0784] The server analyzes the received data packets and inputs them into a generative AI model. The generative AI model extracts facial features (shape, skin tone, eyebrow shape, etc.) from the user's face photo and analyzes the user's body type and style from a full-body photo. It then generates optimal makeup, hairstyles, and outfits based on the situation, preferences, and emotional analysis results.

[0785] Step 5:

[0786] The server then sends the generated makeup, hairstyle, and outfit suggestions to the device. The generated suggestions are formatted in a way that is intuitively understandable to the user.

[0787] Step 6:

[0788] The device then transmits the generated results to a virtual reality device, which can be a head-mounted display or smart glasses, allowing the user to try out the proposed styles in real time in a virtual space.

[0789] Step 7:

[0790] If the user tries out the suggested makeup, hairstyle, and outfit and likes it, they can search for nearby beauty salons and clothing stores through the device, make a reservation, and pay. The device presents these operations to the user as a series of processes.

[0791] Step 8:

[0792] The terminal sends the user's reservation and payment information to the server, which then relays it to the corresponding vendor. Once the server receives the reservation confirmation from the vendor, the server completes the payment process, sends the final confirmation information to the terminal, and notifies the user.

[0793] This series of processes allows users to easily find the makeup, hairstyle, and outfit that best suits them, and they can then check it in real time in the VR space, making reservations and payments smoothly.

[0794] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0795] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0796] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[0797] [Third embodiment]

[0798] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[0799] 5, the data processing system 310 includes the data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[0800] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0801] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[0802] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0803] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0804] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0805] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0806] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0807] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0808] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0809] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[0810] System Overview

[0811] This invention is a system that allows users to receive recommendations for the best makeup, hairstyle, and outfits for themselves, and to make reservations and payments at beauty salons, esthetic salons, and clothing stores all in one place. The system begins when users upload photos of their face and style, and input the situation and preferences. A specific embodiment of this system is described in detail below.

[0812] Program processing flow

[0813] 1. User Input Processing

[0814] When a user launches the application, the device requests the user's account authentication information and prompts the user to log in. After logging in, the user uploads a face photo and a full-body photo. The user also inputs the situation of the day (e.g., date, work, party) and their personal preferences (e.g., natural makeup, casual style).

[0815] 2. Data transmission process

[0816] The device sends the facial photo, full-body photo, situation, and preference information acquired from the user to the server as a single data package. After the transmission is successful, the device notifies the user of the success.

[0817] 3. Data analysis and proposal generation process

[0818] The server analyzes the received data and inputs it into a generative AI model. The generative AI model extracts facial features (shape, skin tone, eyebrow shape, etc.) from the user's face photo and analyzes the user's body type and style from a full-body photo. It then generates optimal makeup, hairstyles, and outfits based on the situation and preferences. The generated suggestions are sent from the server to the device.

[0819] 4. Display of proposed results

[0820] The device analyzes the generated results received from the server and displays them in a format that is easy for the user to understand. The suggestions include the most suitable makeup, hairstyle, and outfit. The user checks the displayed suggestions and, if they like them, proceeds to the next step.

[0821] 5. Booking and payment processing

[0822] If the user likes the suggestions, they use the device's interface to search for nearby beauty salons, esthetic salons, and clothing stores, make a reservation, and pay. The device then sends the reservation information and payment data to the server, which relays it to the appropriate provider. The provider then sends a reservation confirmation to the server, which then processes the successful payment. Finally, the device displays a notification that the reservation and payment have been completed.

[0823] Specific examples

[0824] For example, consider the case where a user uploads a photo of their face and a full body in a situation where they want to know what kind of hairstyle is good for their date. If the user enters "I like casual styles," the system operates as follows:

[0825] 1. User input processing: The user uploads a photo of their face and a full-body photo, and enters the situation and preferences.

[0826] 2. Data transmission process: The device sends a face photo, a full-body photo, the situation, and preference information to the server.

[0827] 3. Data analysis and proposal generation process: The server analyzes the received information and proposes natural curly hair as a "casual hairstyle suitable for a date."

[0828] 4. Displaying the proposal results: The device displays the proposals for natural curly hair to the user.

[0829] 5. Booking and payment processing: The user decides to try the suggested hairstyle, searches for a nearby hair salon, makes a booking and makes payment.

[0830] In this way, users can easily find the makeup, hairstyle, and outfit that suits them, and can smoothly make reservations and make payments. This system provides optimal services to users who are interested in beauty and fashion but don't have much time.

[0831] The processing flow will be explained below.

[0832] Step 1:

[0833] The user launches the application. The device prompts the user for account authentication information, and the user enters the login information.

[0834] Step 2:

[0835] The user takes or uploads a photo of their face and a full-body photo, and then inputs the situation (e.g., date, work, party) and preferences (e.g., natural makeup, casual style).

[0836] Step 3:

[0837] The terminal transmits the face photo, the style photo, the situation, and the preference information to the server as a single data package. After the transmission is successful, the terminal notifies the user of the success.

[0838] Step 4:

[0839] The server inputs the received data into a generative AI model for analysis. The generative AI model extracts facial features (shape, skin tone, eyebrow shape, etc.) from the user's face photo and analyzes the user's body shape and style from a full-body photo.

[0840] Step 5:

[0841] The server generates the optimal makeup, hairstyle, and outfit based on the analysis results and the user's situation and preferences, and sends the generated results to the device.

[0842] Step 6:

[0843] The device analyzes the generated results received from the server and displays them in a format that is easy for the user to understand. The user checks the proposal and, if they like it, proceeds to the next step.

[0844] Step 7:

[0845] The user decides to make a reservation and make payment based on the proposed service. The terminal searches for nearby beauty salons, esthetic salons, and clothing stores and displays a list of businesses that can provide the proposed service.

[0846] Step 8:

[0847] The user selects a vendor and specifies a reservation date and time. After the selection and reservation information are confirmed, the terminal sends the reservation information and payment data to the server.

[0848] Step 9:

[0849] The server relays the reservation information to the merchant and receives confirmation of the reservation. The server processes the payment using PayPay and confirms the payment was successful.

[0850] Step 10:

[0851] The server sends a notification of completion of reservation and payment to the terminal. The terminal notifies the user that reservation and payment have been completed.

[0852] This series of processing flows allows users to easily and quickly receive suggestions for optimal makeup, hairstyles, and outfits, while also smoothly making reservations and making payments.

[0853] Example 1

[0854] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[0855] Previously, to receive beauty and fashion recommendations, users had to search for multiple services and stores individually, make reservations, and pay. This was time-consuming and labor-intensive, making it difficult to receive optimal recommendations. Furthermore, technology to automatically generate recommendations tailored to a user's preferences and situation was insufficient, limiting the user experience.

[0856] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0857] In this invention, the server includes a means for inputting prompt text into the generative AI model to suggest makeup, hairstyles, and outfits, a means for the server to transmit the generated results to a terminal, and a means for the user to make reservations and payments based on the generated results. This allows users to receive optimal beauty and fashion suggestions simply by uploading photos of their face and style and inputting the situation and preferences. Furthermore, reservations and payments can be made all at once, significantly reducing the user's time and effort.

[0858] "User" refers to an individual or organization that uses the system.

[0859] "Terminal" refers to the device used by a User (e.g., smartphone, tablet, computer).

[0860] A "face photo" refers to an image of the user's face.

[0861] A "style photo" refers to an image of the user's entire body.

[0862] A "situation" refers to a specific event or purpose (e.g., date, work, party) entered by the user.

[0863] "Preferences" refers to the preferences regarding fashion and makeup input by the user (e.g., natural makeup, casual style).

[0864] "Server" refers to a computer system that processes, stores, and transmits data.

[0865] A "data package" refers to a collection of data including a face photo, style photo, situation, and preference information.

[0866] A "generative AI model" refers to an artificial intelligence algorithm that suggests makeup, hairstyles, and outfits based on input data.

[0867] A "prompt" refers to a specific instruction or request given to a generative AI model.

[0868] "Suggestion generation" refers to the process by which the generative AI model generates the best makeup, hairstyle, and outfit for the user based on the analysis results.

[0869] "Reservation" refers to the act of a user setting a date and time for using services at a beauty salon, esthetic salon, or clothing store based on the content suggested by the user.

[0870] "Payment" refers to the act of paying monetary consideration for the use of a service.

[0871] "Businesses" refer to service providers such as hair salons, beauty salons, and clothing stores.

[0872] "Generated results" refers to the makeup, hairstyle, and outfit suggestions generated by the generative AI model.

[0873] "Relay" refers to the act of the server forwarding information received from the user to the appropriate provider.

[0874] This invention is a system that allows users to receive suggestions for the best makeup, hairstyle, and outfits for themselves, and to make reservations and payments at beauty salons, esthetic salons, and clothing stores all in one place. This system is realized mainly using a terminal, a server, and a generative AI model.

[0875] Specific program description

[0876] User Input Processing

[0877] A user launches the application and uses their device to enter their account credentials to log in. After successfully logging in, the user uploads a face photo and a full-body photo to the device. In addition, the user inputs the specific situation (e.g., date, work, party) and their preferences (e.g., natural makeup, casual style), which allows the system to obtain the user's specific needs.

[0878] Data transmission process

[0879] The information entered by the user (face photo, full-body photo, situation, preferences) is sent as a single data package from the device to the server. The device notifies the user of the success of the data transmission.

[0880] Data analysis and proposal generation process

[0881] The server analyzes the received data package and inputs it as a prompt to the generative AI model. The generative AI model extracts facial features (shape, skin tone, eyebrow shape, etc.) from the user's face photo and analyzes body shape and style from a full-body photo. It also suggests the optimal makeup, hairstyle, and outfit based on the situation and preferences. The suggestions are sent from the server to the device.

[0882] Prompt Sentence Examples

[0883] "User uploaded a photo for a date. She prefers a casual look. Please suggest the best makeup and hairstyle for her."

[0884] Display of proposal results

[0885] The device analyzes the generated results received from the server and displays the proposals in a format that is easy for the user to understand. The user can review the proposals and, if they like them, proceed to the next step.

[0886] Reservation and payment processing

[0887] If the user likes the suggestions, they can search for nearby beauty salons, esthetic salons, and clothing stores via their device, make a reservation, and make payment. The reservation information and payment data are sent from the device to the server, which relays this information to the appropriate provider. After the server receives a reservation confirmation from the provider, the server processes the successful payment and displays a notification on the device that the reservation and payment have been completed.

[0888] Specific examples

[0889] For example, consider the case where a user uploads a photo of their face and a full body in a situation where they want to know what kind of hairstyle is good for their date. If the user enters "I like casual styles," the system operates as follows:

[0890] 1. User input processing: The user uploads a photo of their face and a full-body photo, and enters the situation and preferences.

[0891] 2. Data transmission process: The device sends a face photo, a full-body photo, the situation, and preference information to the server.

[0892] 3. Data analysis and proposal generation process: The server analyzes the received information and proposes natural curly hair as a "casual hairstyle suitable for a date."

[0893] 4. Displaying the proposal results: The device displays the proposals for natural curly hair to the user.

[0894] 5. Booking and payment processing: The user decides to try the suggested hairstyle, searches for a nearby hair salon, books a reservation and makes payment.

[0895] In this way, users can easily find the makeup, hairstyle, and outfit that suits them, and can smoothly make reservations and make payments. This system can provide the best service, especially to users who are interested in beauty and fashion but are short on time.

[0896] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0897] Specific explanation of processing steps

[0898] Step 1:

[0899] The user launches the application and goes to the login screen. The user enters their account information (username, password) and clicks the login button. The device sends this authentication information as input to the server. The server receives the authentication information and compares it with a database to determine whether the authentication was successful. If the authentication is successful, the server returns the information to the device, and the device outputs the home screen.

[0900] Step 2:

[0901] The user takes or selects a face photo and a full-body photo within the application and clicks the upload button. The device temporarily stores these photo data in memory. Next, a form is displayed in which the user can input the situation (e.g., date, work, party) and preferences (e.g., natural makeup, casual style). The user enters this information and clicks the submit button. These inputs are then compiled into a single data package.

[0902] Step 3:

[0903] The device sends the compiled data package (face photo, full-body photo, situation, and preference information) as input to the server. The server receives this data package and confirms its receipt. After confirmation, the server outputs a notification to the device that reception is complete. The device then notifies the user that the data transmission was successful.

[0904] Step 4:

[0905] The server analyzes the received data package. This analysis process uses facial recognition software (e.g., OpenCV) and data analysis software (e.g., TensorFlow). The server first extracts facial features (e.g., face shape, skin tone, eyebrow shape, etc.) from the face photo, and then analyzes body shape and style from the full-body photo. Next, it generates a prompt based on the situation and preferences. An example prompt is, "Please suggest makeup and hairstyle that suits this face photo and situation."

[0906] Step 5:

[0907] The server inputs the generated prompt into the generative AI model, which then suggests the best makeup, hairstyle, and outfit for the user based on the prompt. The generative AI model returns the generated suggestions (e.g., natural curly hair, beige trench coat, denim pants) to the server, which then receives them and sends them to the device.

[0908] Step 6:

[0909] The device analyzes the generated results received from the server and displays the suggestions in a user-friendly format. This allows the user to see specific makeup, hairstyle, and outfit suggestions. If the user is satisfied with the suggestions, they can proceed to the next step based on the suggestions.

[0910] Step 7:

[0911] If the user wants to try out the suggested service, they use the device's interface to search for nearby beauty salons, esthetic salons, and clothing stores. The device uses the user's current location information as input. The user selects the desired store from the displayed list and sets the reservation date and time. When the user clicks the reservation button, the device sends this reservation information to the server.

[0912] Step 8:

[0913] The server processes the received reservation information and relays it to the appropriate vendor. Once the server receives confirmation of the reservation from the vendor, the server initiates the payment process and notifies the terminal of successful payment. Finally, the terminal displays a notification to the user confirming the reservation and payment.

[0914] This allows users to easily find makeup, hairstyles, and outfits that suit them, and also makes reservations and payments smoothly.

[0915] (Application example 1)

[0916] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[0917] In conventional beauty consulting and service booking systems, it was difficult for users to find the best makeup and hairstyle for themselves, and it was also difficult to smoothly book and pay with service providers, which caused many users to feel tedious and time-consuming.

[0918] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0919] In this invention, the server includes means for generating makeup, hairstyles, and outfits based on a user's facial photograph and style photograph using a generative AI model, means for transmitting the generated results to the terminal, and means for searching for and selecting a service provider for which a reservation has been made. This allows the user to easily find the makeup, hairstyle, and outfit that best suits them, and further allows for smooth booking and payment of related services all at once.

[0920] "User" refers to an individual who uses the system to upload a photo of their face and style, and input the situation and preferences.

[0921] "Mouthshot" refers to a photograph used to clearly show a user's facial features.

[0922] A "style photo" refers to a photo that shows the user's entire body and is used to show features such as body shape and clothing.

[0923] "Situation" refers to a specific scene or purpose for the makeup, hairstyle, and coordination desired by the user.

[0924] "Preferences" refer to the style and trends of makeup, hairstyle, and outfits desired by the user.

[0925] "Terminal" refers to an electronic device that a user uses to upload photos, input information, and communicate with a server.

[0926] "Server" refers to the central processing unit that receives and analyzes information sent by the user, generates optimal makeup, hairstyles, and outfits using a generative AI model, and sends the results to the terminal.

[0927] "Analysis" refers to the process by which the server analyzes the face photo, style photo, situation, and preference information it receives and makes appropriate suggestions based on that information.

[0928] A "generative AI model" refers to an artificial intelligence model that uses machine learning algorithms to analyze a user's characteristics and suggest optimal makeup, hairstyles, and outfits.

[0929] "Makeup, hairstyle, and coordination" refers to specific beauty and fashion suggestions that AI suggests based on the user's face photo, style photo, situation, and preferences.

[0930] "Generated results" refers to the makeup, hairstyle, and outfit suggestions generated by the generative AI model.

[0931] "Display" refers to the process by which the terminal visually shows the generated results to the user.

[0932] "Reservation" refers to a user making a reservation with a specific service provider based on the makeup, hairstyle, and outfit suggestions.

[0933] "Payment" refers to the process of paying for the service a user has reserved.

[0934] "Service provider" refers to a business that provides beauty and fashion-related services to users, such as a beauty salon, esthetic salon, or clothing store.

[0935] "Search" refers to the process performed by a terminal to find a particular service provider based on a user request.

[0936] "Selection" refers to the act of a user selecting a particular service provider from the search results.

[0937] System Overview

[0938] This invention is a system that allows users to receive recommendations for the best makeup, hairstyle, and outfits for themselves, and then make reservations and payments with on-site service providers all in one place. The system begins by users uploading photos of their face and style using a terminal, and inputting the situation and preferences. A specific embodiment of this system is described in detail below.

[0939] Program processing flow

[0940] User Input Processing

[0941] When a user launches the application, the device requests the user's account authentication information and prompts the user to log in. After logging in, the user uploads a face photo and a full-body photo, and enters the situation of the day (e.g., date, work, party) and their preferences (e.g., natural makeup, casual style).

[0942] Data transmission process

[0943] The device sends the facial photo, full-body photo, situation, and preference information acquired from the user to the server as a single data package. After the transmission is successful, the device notifies the user of the success.

[0944] Data analysis and proposal generation process

[0945] The server analyzes the received data and inputs it into a generative AI model. The generative AI model extracts facial features (shape, skin tone, eyebrow shape, etc.) from the user's face photo and analyzes the user's body type and style from a full-body photo. It then generates optimal makeup, hairstyles, and outfits based on the situation and preferences. The generated suggestions are sent from the server to the device.

[0946] Display of proposal results

[0947] The device analyzes the generated results received from the server and displays them in a format that is easy for the user to understand. The suggestions include the most suitable makeup, hairstyle, and outfit. The user checks the displayed suggestions and, if they like them, proceeds to the next step.

[0948] Reservation and payment processing

[0949] If the user likes the suggestions, they use the terminal interface to search for relevant service providers (e.g., hair salons, esthetic salons, clothing stores, etc.) and make a reservation and payment. The terminal sends the reservation information and payment data to the server, which relays it to the appropriate provider. The provider's reservation confirmation arrives at the server, which processes the successful payment. Finally, the terminal displays a notification that the reservation and payment have been completed.

[0950] Example of a system

[0951] When applying this system as a smartphone application, the following specific examples can be considered.

[0952] Examples:

[0953] For example, consider the case where a user uploads a photo of their face and a full body in a situation where they want to know what kind of hairstyle is good for their date. If the user enters "I like casual styles," the system operates as follows:

[0954] 1. The user uploads a photo of their face and a full-body photo, and enters the situation and preferences.

[0955] 2. The device sends a photo of the face, a full-body photo, the situation, and preference information to the server.

[0956] 3. The server analyzes the received information and uses a generative AI model to suggest natural curls as a "casual hairstyle suitable for a date."

[0957] 4. The device will display natural curl suggestions to the user.

[0958] 5. The user decides to try the suggested hairstyle and searches for a nearby hair salon, makes a reservation, and pays.

[0959] In this way, users can easily find the makeup, hairstyle, and outfit that suits them, and can also smoothly book and pay for related services.

[0960] Example prompts to input to a generative AI model:

[0961] "Please extract the user's features from face and full-body photos, and suggest the best makeup, hairstyle, and outfit based on the situation (date) and preference (casual)."

[0962] This system makes it possible to provide optimal services to users who are interested in beauty and fashion but don't have much time.

[0963] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0964] Step 1: User Input Processing

[0965] A user launches the application and enters their account authentication information on the login screen. After logging in, they upload a face photo and a full-body photo, and enter the situation of the day (e.g., date, work, party) and their preferences (e.g., natural makeup, casual style). This allows the user's characteristic information to be collected. The input data includes the face photo, full-body photo, situation, and preference information.

[0966] Step 2: Data transmission process

[0967] The device sends the face photo, full-body photo, situation, and preference information acquired from the user to the server as a single data package. The sent data consists of an image file and text information. When the server receives the data, it notifies the device of the success. The device then notifies the user of the success of the data transmission.

[0968] Step 3: Data analysis and proposal generation process

[0969] The server inputs the received data into a generative AI model for analysis. The generative AI model extracts facial features (e.g., facial contours, skin tone, eyebrow shape, etc.) from the face photo and analyzes body shape and clothing characteristics from the full-body photo. Next, it generates optimal makeup, hairstyle, and outfits based on the situation and preferences. The generated suggestions consist of specific advice on makeup, hairstyle, and outfits. Finally, these suggestions are sent from the server to the device.

[0970] Step 4: Displaying the proposed results

[0971] The terminal analyzes the generated results received from the server and displays them in a format that is easy for the user to understand. They are displayed in a specific screen layout (for example, images and descriptions of makeup, hairstyles, and outfits). The user checks these suggestions and, if they like them, proceeds to the next step. The input is the generated results from the server, and the output is the suggestions that the user visually checks.

[0972] Step 5: Booking and payment processing

[0973] If the user likes the proposed service, they use the terminal interface to search for nearby service providers (e.g., hair salons, esthetic salons, clothing stores, etc.). The user selects the desired service provider from the search results and makes a reservation and makes a payment. The terminal sends the reservation information and payment data as a single data package to the server. The server relays the received information and transmits the reservation information to the appropriate service provider. The server also receives reservation confirmation from the provider and notifies the user's terminal. The user can finally confirm that the reservation and payment have been completed. The input is the reservation and payment information, and the output is a notification that the reservation and payment have been completed.

[0974] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0975] System Overview

[0976] This invention is a system that allows users to receive recommendations for the best makeup, hairstyle, and outfits for themselves, and to make reservations and payments at beauty salons, esthetic salons, and clothing stores all in one place. Furthermore, this invention incorporates an emotion engine that analyzes the user's emotions, making it possible to make optimal recommendations based on the user's emotions. This system begins when the user uploads a photo of their face and style, and inputs the situation and preferences. Specific embodiments of this system are described in detail below.

[0977] Program processing flow

[0978] 1. User Input Processing

[0979] When a user launches the application, the device prompts the user for account authentication information, and the user enters the login information. After logging in, the user takes or uploads a face photo and a full-body photo. The user also enters the situation of the day (e.g., date, work, party) and their preferences (e.g., natural makeup, casual style).

[0980] 2. Emotion analysis processing

[0981] The device sends a facial photo to the emotion engine, which then analyzes the user's emotions from the facial photo. The emotion engine recognizes emotions such as happiness, surprise, sadness, and anger, and returns the analysis results to the device. The device then sends the emotion analysis results to the server.

[0982] 3. Data transmission process

[0983] The device sends the facial photo, full-body photo, situation, preference information, and emotion analysis results acquired from the user to the server as a single data package. After the transmission is successful, the device notifies the user of the success.

[0984] 4. Data analysis and proposal generation process

[0985] The server inputs the received data into a generative AI model for analysis. The generative AI model extracts facial features (shape, skin tone, eyebrow shape, etc.) from the user's face photo and analyzes the user's body type and style from a full-body photo. It then generates optimal makeup, hairstyles, and outfits based on the situation, preferences, and emotional analysis results. The generated suggestions are sent from the server to the device.

[0986] 5. Display of proposed results

[0987] The device analyzes the generated results received from the server and displays them in a format that is easy for the user to understand. The suggestions include the most suitable makeup, hairstyle, and outfit. The user reviews the suggestions and, if they like them, proceeds to the next step.

[0988] 6. Booking and payment processing

[0989] If the user likes the suggestions, they use the device's interface to search for nearby beauty salons, esthetic salons, and clothing stores, make a reservation, and pay. The device then sends the reservation information and payment data to the server, which relays it to the appropriate provider. The provider then sends a reservation confirmation to the server, which then processes the successful payment. Finally, the device displays a notification that the reservation and payment have been completed.

[0990] Specific examples

[0991] For example, consider a situation where a user uploads a photo of their face and a full body in a situation where they want to know what kind of hairstyle is best for their date. If the user enters "I like casual styles," the emotion engine analyzes the emotion of "happiness" from the user's face. The system works as follows:

[0992] 1. User input processing: The user uploads a photo of their face and a full-body photo, and enters the situation and preferences.

[0993] 2. Emotion analysis processing: The emotion engine analyzes the facial photo, recognizes the emotion of happiness, and sends the results back to the device.

[0994] 3. Data transmission process: The device sends a face photo, a full-body photo, the situation, preferences, and emotion analysis results to the server.

[0995] 4. Data analysis and proposal generation process: Based on the analysis results, the server proposes natural curly hair as a hairstyle suitable for a date that corresponds to casual and happy emotions.

[0996] 5. Displaying the proposal results: The device displays the proposals for natural curly hair to the user.

[0997] 6. Booking and payment processing: If the user likes the suggested hairstyle, they can search for nearby salons, make a booking and make a payment.

[0998] This series of processes allows users to easily find makeup, hairstyles, and outfits that suit them, and also allows for smooth booking and payment. By combining it with an emotion engine, more personalized suggestions can be made based on the user's emotions.

[0999] The processing flow will be explained below.

[1000] Step 1:

[1001] The user launches the application. The device prompts the user for account authentication information, and the user enters the login information.

[1002] Step 2:

[1003] The user takes or uploads a photo of their face and a full-body photo, and then inputs the situation (e.g., date, work, party) and preferences (e.g., natural makeup, casual style).

[1004] Step 3:

[1005] The device sends a facial photo to the emotion engine, which analyzes the user's facial photo and recognizes emotions such as happiness, surprise, sadness, and anger. The analysis results are then sent back to the device.

[1006] Step 4:

[1007] The terminal transmits the face photo, the style photo, the situation, the preference information, and the emotion analysis result to the server as a single data package. After the transmission is successful, the terminal notifies the user of the success.

[1008] Step 5:

[1009] The server inputs the received data into a generative AI model for analysis. The generative AI model extracts facial features (shape, skin tone, eyebrow shape, etc.) from the user's face photo and analyzes the user's body shape and style from a full-body photo.

[1010] Step 6:

[1011] The server generates the optimal makeup, hairstyle, and outfit based on the analysis results, situation, preferences, and emotion analysis results, and then sends the generated results to the device.

[1012] Step 7:

[1013] The device analyzes the generated results received from the server and displays them in an easy-to-understand format for the user. The user checks the suggestions and, if they like them, proceeds to the next step.

[1014] Step 8:

[1015] If the user likes the suggestions, they can use the device's interface to search for nearby beauty salons, esthetic salons, and clothing stores, and a list of businesses that can provide the suggested services will be displayed on the device.

[1016] Step 9:

[1017] The user selects a vendor and specifies the reservation date and time on a new reservation screen. After the selection and reservation information are confirmed, the terminal sends the reservation information and payment data to the server.

[1018] Step 10:

[1019] The server relays the reservation information to the vendor and receives confirmation of the reservation. The server processes the payment using PayPay and confirms the payment was successful.

[1020] Step 11:

[1021] The server sends a notification of completion of reservation and payment to the terminal. The terminal notifies the user that reservation and payment have been completed.

[1022] This detailed process flow allows users to easily find the makeup, hairstyle, and outfit that suits them, and also allows them to receive personalized suggestions based on emotion analysis. In addition, reservations and payments can be made smoothly, providing optimal service even to busy users.

[1023] Example 2

[1024] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1025] Conventional beauty recommendation systems have the problem that it takes a lot of time and effort for users to find the makeup, hairstyle, and coordination that is best suited to them. Furthermore, since the recommendations do not reflect the user's emotions or situation, the recommendations provided are difficult to personalize. Furthermore, users have to make separate reservations and payments at beauty salons, esthetic salons, and clothing stores based on the recommendations, making the process cumbersome.

[1026] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1027] In this invention, the server includes: means for a user to upload a face photo and a style photo; means for a user to input a situation and preferences; means for a terminal to transmit the face photo, the style photo, the situation, and the preference information to the server; means for the server to input the received information into a generative AI model, analyze facial features and style, and generate optimal makeup, hairstyle, and outfits; means for the terminal to transmit the face photo to an emotion analysis engine and analyze emotions; means for transmitting the emotion analysis results to the server; means for the server to generate a proposal using the emotion analysis results; means for the server to transmit the generation results to the terminal; means for the terminal to display the generation results to the user; means for the user to make a reservation and payment based on the generation results; and means for the server to process the reservation and payment. This allows a user to quickly receive optimal beauty suggestions personalized to them and further allows them to make a reservation and payment based on the proposals all at once.

[1028] "User" refers to any individual or organization that uses this system.

[1029] "Terminal" refers to an electronic device (e.g., a smartphone or PC) operated by a user.

[1030] A "server" refers to a computer that manages the entire system, processes and stores data, and runs generative AI models.

[1031] "Face photo" refers to an image file of a user's face.

[1032] A "style photo" refers to an image file that captures the user's entire body.

[1033] "Situation" refers to the situation or event of the day (e.g., date, work, party) entered by the user.

[1034] "Preferences" refers to personal style and makeup preferences entered by the user (e.g., natural makeup, casual style).

[1035] A "generative AI model" refers to an artificial intelligence model that generates optimal makeup, hairstyles, and outfits based on input data.

[1036] An "emotion analysis engine" refers to software that analyzes emotions from a user's facial photo and provides the results.

[1037] "Data package" refers to a data set that compiles user-entered facial photos, style photos, situations, preference information, and emotion analysis results.

[1038] A "reservation" refers to a user setting a time and date at a store such as a beauty salon, esthetic salon, or clothing store to receive the proposed beauty service.

[1039] "Payment" means the online payment of the service booked by the User.

[1040] MODE FOR CARRYING OUT THE INVENTION

[1041] This invention is a system that allows users to receive recommendations for the best makeup, hairstyle, and outfits for themselves, and to make reservations and payments at hair salons, esthetic salons, and clothing stores all in one place. Furthermore, by incorporating an emotion analysis engine, it is possible to make optimal recommendations based on the user's emotions. This system mainly uses the following hardware and software:

[1042] Hardware and software used

[1043] Device: Electronic device operated by the user, such as a smartphone or computer

[1044] Server: A computer that manages the entire system, processes and stores data, and runs generative AI models.

[1045] Generative AI model: An artificial intelligence model that generates optimal makeup, hairstyles, and outfits based on user information (e.g., "StyleAI")

[1046] Emotion analysis engine: Software that analyzes emotions from a user's facial photo and provides the results (e.g., "EmotionAI")

[1047] Specific examples of programs

[1048] The system starts by having users upload a photo of their face and style, and input the situation and preferences.

[1049] 1. User Input Processing

[1050] When a user launches the application, the device prompts the user for account authentication information, and the user enters the login information. After successful login, the user takes or uploads a face photo and a full-body photo, and enters the situation (e.g., date, work, party) and preferences (e.g., natural makeup, casual style).

[1051] 2. Emotion analysis processing

[1052] The device sends a facial photo to the emotion analysis engine, which then analyzes the user's emotions from the facial photo. The emotion analysis engine recognizes emotions such as happiness, surprise, sadness, and anger, and returns the analysis results to the device. The device then sends the emotion analysis results to the server.

[1053] 3. Data transmission process

[1054] The device sends the facial photo, full-body photo, situation, preference information, and emotion analysis results acquired from the user to the server as a single data package. After the transmission is successful, the device notifies the user of the success.

[1055] 4. Data analysis and proposal generation process

[1056] The server inputs the received data into a generative AI model. The generative AI model extracts facial features (shape, skin tone, eyebrow shape, etc.) from the user's face photo and analyzes the user's body type and style from a full-body photo. It then generates optimal makeup, hairstyles, and outfits based on the situation, preferences, and emotional analysis results. The generated suggestions are sent from the server to the device.

[1057] 5. Display of proposed results

[1058] The device analyzes the generated results received from the server and displays them in a format that is easy for the user to understand. The suggestions include the most suitable makeup, hairstyle, and outfit. The user reviews the suggestions and, if they like them, proceeds to the next step.

[1059] 6. Booking and payment processing

[1060] If the user likes the suggestions, they use the device's interface to search for nearby beauty salons, esthetic salons, and clothing stores, make a reservation, and pay. The device then sends the reservation information and payment data to the server, which relays it to the appropriate provider. The provider then sends a reservation confirmation to the server, which then processes the successful payment. Finally, the device displays a notification that the reservation and payment have been completed.

[1061] Specific examples

[1062] For example, if a user uploads a photo of their face and a full body photo in a situation where they have a date and want to know what kind of hairstyle is nice, the following will happen.

[1063] Users upload photos of their face and body, and enter the situation and preferences.

[1064] The emotion analysis engine analyzes the facial photo, recognizes the emotion of happiness, and sends the results back to the device, which then sends the emotion analysis results to the server.

[1065] The device sends a photo of the face, a full-body photo, the situation, preferences, and emotional analysis results to the server.

[1066] The server inputs the data into a generative AI model, which suggests natural curls as a hairstyle suitable for a date, corresponding to casual and happy feelings.

[1067] The device displays natural curl suggestions to the user.

[1068] The user likes the suggested hairstyle, searches for a nearby hair salon, makes a reservation, and makes payment.

[1069] This series of processes allows users to easily find the makeup, hairstyle, and outfit that suits them, and also allows for smooth booking and payment. By combining it with a sentiment analysis engine, more personalized suggestions can be made based on the user's emotions.

[1070] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1071] Program processing flow

[1072] Step 1: User Input Processing

[1073] Input: User login information, face photo, full-body photo, situation, preferences

[1074] Output: Face photo, full body photo, situation, preference data package

[1075] Specific behavior:

[1076] 1. The user launches the application and enters login information (username and password) into the terminal.

[1077] 2. The device sends the login information to the server for authentication. If the login is successful, the device displays a screen for the user to take or upload a face photo and a full-body photo.

[1078] 3. The user uploads a face photo and a full-body photo. The device then displays a situation input screen, and the user selects an appropriate situation (e.g., date, work, party).

[1079] 4. Next, the device displays a preference input screen, and the user selects their preference (e.g., natural makeup, casual style).

[1080] Step 2: Sentiment Analysis Processing

[1081] Input: Face photo

[1082] Output: Emotion analysis results

[1083] Specific behavior:

[1084] 1. The device sends a photo of the user's face to an emotion analysis engine (e.g., "EmotionAI").

[1085] 2. The emotion analysis engine analyzes the facial photo, recognizes emotions such as happiness, surprise, sadness, and anger, and sends the results back to the device.

[1086] 3. The device receives the emotion analysis results and sends them to the server.

[1087] Step 3: Data transmission process

[1088] Input: Face photo, full-body photo, situation, preferences, emotion analysis results

[1089] Output: Send data package to server

[1090] Specific behavior:

[1091] 1. The device compiles the facial photo, full-body photo, situation, preference information, and emotion analysis results obtained from the user into a single data package.

[1092] 2. The device sends this data package to the server, and after successful transmission, the device notifies the user of the success.

[1093] Step 4: Data analysis and proposal generation process

[1094] Input: Data package (face photo, full-body photo, situation, preferences, emotion analysis results)

[1095] Output: Suggestions (optimal makeup, hairstyle, outfit)

[1096] Specific behavior:

[1097] 1. The server inputs the received data package into a generative AI model (e.g., "StyleAI") for analysis.

[1098] 2. The generative AI model extracts facial features (shape, skin tone, eyebrow shape, etc.) from a photo of the user's face and analyzes the user's body shape and style from a full-body photo.

[1099] 3. The server generates the optimal makeup, hairstyle, and outfit based on the situation, preferences, and emotional analysis results.

[1100] 4. The generated proposal is sent from the server to the device.

[1101] Step 5: Displaying the proposed results

[1102] Input: Suggestion details (best makeup, hairstyle, outfit)

[1103] Output: Suggestion results to be displayed

[1104] Specific behavior:

[1105] 1. The terminal analyzes the generated results received from the server and displays them in a format that is easy for the user to understand.

[1106] 2. The suggestions include the best makeup, hairstyle, and outfit.

[1107] 3. The user reviews the suggestions and, if they like them, proceeds to the next step.

[1108] Step 6: Booking and payment processing

[1109] Input: Proposal details, reservation information, payment information

[1110] Output: Notification of reservation and payment completion

[1111] Specific behavior:

[1112] 1. If the user likes the suggestions, the device provides an interface for searching for nearby hair salons, beauty salons, and clothing stores.

[1113] 2. The user selects the desired store and enters reservation information (date, time, service details, etc.).

[1114] 3. The terminal prompts the user to enter payment information (such as credit card information), after which the terminal sends the reservation information and payment data to the server.

[1115] 4. The server relays the reservation information and payment data to the appropriate vendor, and the vendor sends a confirmation of the reservation to the server.

[1116] 5. The server processes the successful payment and finally displays a notification on the terminal that the reservation and payment are complete.

[1117] (Application example 2)

[1118] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1119] While conventional beauty recommendation systems can suggest optimal makeup, hairstyles, and outfits to users, they face the challenge of making it difficult to actually try out and confirm the suggestions. Furthermore, when making reservations and paying at beauty salons or clothing stores based on the suggestions, multiple steps are required, which is cumbersome for users. This results in limited accuracy of the suggestions and a poor user experience.

[1120] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for generating makeup, hairstyles, and outfits based on the user's face photo, style photo, situation, and preferences using a generative AI model, means for transmitting the generated results to the terminal, and means for the terminal to have the user try on the generated results using a virtual reality device. This allows the user to intuitively check the proposed styles in real time, and also enables reservations and payments to be made all at once.

[1121] "User" refers to an individual who uses this system.

[1122] "Facial photo" refers to image data of the user's face.

[1123] A "style photo" refers to image data that shows the user's entire body.

[1124] "Situation" refers to the purpose or situation in which a user uses the system.

[1125] "Preferences" refer to the fashion and makeup styles desired by the user.

[1126] "Terminal" refers to a device used to use this system, such as a smartphone, tablet, or PC.

[1127] "Server" refers to a remote computer system that communicates with and processes data from a terminal.

[1128] A "generative AI model" refers to an artificial intelligence algorithm that suggests optimal makeup, hairstyles, and outfits based on a user's face photo, style photo, situation, and preferences.

[1129] "Generated results" refers to the makeup, hairstyle, and outfit suggestions obtained using the generative AI model.

[1130] "Means for displaying" refers to the function of visually showing the generated results on the terminal screen.

[1131] "Reservation" refers to the act of a user reserving services in advance at a beauty salon, clothing store, or the like in order to actually perform the suggested makeup or hairstyle.

[1132] "Payment" refers to the act of paying in advance for the service a user uses.

[1133] "Virtual reality device" refers to equipment (e.g., a head-mounted display or smart glasses) that allows users to try out generated results in a virtual space.

[1134] "Trying on" refers to a user virtually experiencing suggested makeup, hairstyles, and outfits using a virtual reality device.

[1135] This invention is a system that allows users to receive personalized recommendations for their best makeup, hairstyle, and outfits, and to make reservations and payments at beauty salons and clothing stores all in one place. The system combines an emotion analysis engine with a generative AI model and uses virtual reality devices to enhance the user experience.

[1136] System Overview

[1137] The server receives the face and style photos uploaded by the user, as well as the situation and preference information, and uses a generative AI model to generate the optimal makeup, hairstyle, and outfit.The generated results are then sent to the device and displayed to the user.

[1138] The terminal sends a photo of the user's face and style to the server in response to user operation, receives the generated results, and displays them. The user can also try out the generated results using a virtual reality device. Furthermore, the user can make reservations and make payments at nearby beauty salons or clothing stores based on the generated results they like.

[1139] Hardware and software used

[1140] Hardware:

[1141] Devices such as smartphones, tablets, and computers

[1142] Virtual reality devices (e.g., head-mounted displays, smart glasses)

[1143] software:

[1144] Emotion Analysis Engine: An algorithm for analyzing emotions from user facial photos

[1145] Generative AI model: An artificial intelligence that generates optimal makeup, hairstyles, and outfits based on information submitted by the user.

[1146] API: Manages the sending and receiving of data between the device and the server

[1147] Data processing and calculation

[1148] The server processes the following data:

[1149] Receives a photo of your face and style, and uses image analysis algorithms to extract facial features (shape, skin tone, eyebrow shape, etc.) and body shape

[1150] The situation (date, work, party, etc.), preferences (natural makeup, casual style, etc.) and emotion analysis results (happiness, surprise, sadness, anger, etc.) are taken into account and input into the generative AI model.

[1151] The generative AI model generates optimal makeup, hairstyle, and outfit suggestions, and sends the results to the device.

[1152] Specific examples

[1153] For example, consider a situation where a user uploads a photo of their face and a full body in a situation where they want to know what kind of hairstyle is best for their date. If the user enters "I like casual styles," and the emotion engine analyzes the emotion "happy" from the user's face, the server will perform the following steps:

[1154] Using a generative AI model based on the situation, preferences, and emotional analysis, the app suggests hairstyles suitable for casual dates.

[1155] The proposed natural curls are sent to the device, which then displays them to the user.

[1156] The device uses a virtual reality device to allow the user to try on suggested hairstyles.

[1157] If the user likes the suggested hairstyle, they can search for nearby hair salons, make a reservation, and pay in one go.

[1158] Prompt Sentence Examples

[1159] An example of a prompt for a generative AI model is:

[1160] "Please suggest the best hairstyle based on the user's face photo, full-body photo, situation (date), preference (casual style), and emotion (happiness)."

[1161] In this way, users can easily find the makeup, hairstyle, and outfit that best suits them, and can check it in real time in the VR space, making reservations and payments smoothly.

[1162] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1163] Step 1:

[1164] The user takes or uploads a photo of their face and whole body, and inputs the situation (e.g., date, work, party) and preferences (e.g., natural makeup, casual style). This information is then added to the device as input data.

[1165] Step 2:

[1166] The device sends a facial photo to an emotion analysis engine, which analyzes the user's emotions (e.g., happiness, surprise, sadness, anger). Specifically, a facial recognition algorithm is used to extract emotions from the image and provide the analysis results.

[1167] Step 3:

[1168] The emotion analysis engine returns the emotion analysis results to the terminal, which then sends these analysis results along with other input data (face photo, full-body photo, situation, preferences, etc.) to the server as a packet.

[1169] Step 4:

[1170] The server analyzes the received data packets and inputs them into a generative AI model. The generative AI model extracts facial features (shape, skin tone, eyebrow shape, etc.) from the user's face photo and analyzes the user's body type and style from a full-body photo. It then generates optimal makeup, hairstyles, and outfits based on the situation, preferences, and emotional analysis results.

[1171] Step 5:

[1172] The server then sends the generated makeup, hairstyle, and outfit suggestions to the device. The generated suggestions are formatted in a way that is intuitively understandable to the user.

[1173] Step 6:

[1174] The device then transmits the generated results to a virtual reality device, which can be a head-mounted display or smart glasses, allowing the user to try out the proposed styles in real time in a virtual space.

[1175] Step 7:

[1176] If the user tries out the suggested makeup, hairstyle, and outfit and likes it, they can search for nearby beauty salons and clothing stores through the device, make a reservation, and pay. The device presents these operations to the user as a series of processes.

[1177] Step 8:

[1178] The terminal sends the user's reservation and payment information to the server, which then relays it to the corresponding vendor. Once the server receives the reservation confirmation from the vendor, the server completes the payment process, sends the final confirmation information to the terminal, and notifies the user.

[1179] This series of processes allows users to easily find the makeup, hairstyle, and outfit that best suits them, and they can then check it in real time in the VR space, making reservations and payments smoothly.

[1180] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1181] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1182] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[1183] [Fourth embodiment]

[1184] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[1185] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1186] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1187] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[1188] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1189] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1190] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1191] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[1192] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1193] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1194] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1195] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1196] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1197] System Overview

[1198] This invention is a system that allows users to receive recommendations for the best makeup, hairstyle, and outfits for themselves, and to make reservations and payments at beauty salons, esthetic salons, and clothing stores all in one place. The system begins when users upload photos of their face and style, and input the situation and preferences. A specific embodiment of this system is described in detail below.

[1199] Program processing flow

[1200] 1. User Input Processing

[1201] When a user launches the application, the device requests the user's account authentication information and prompts the user to log in. After logging in, the user uploads a face photo and a full-body photo. The user also inputs the situation of the day (e.g., date, work, party) and their personal preferences (e.g., natural makeup, casual style).

[1202] 2. Data transmission process

[1203] The device sends the facial photo, full-body photo, situation, and preference information acquired from the user to the server as a single data package. After the transmission is successful, the device notifies the user of the success.

[1204] 3. Data analysis and proposal generation process

[1205] The server analyzes the received data and inputs it into a generative AI model. The generative AI model extracts facial features (shape, skin tone, eyebrow shape, etc.) from the user's face photo and analyzes the user's body type and style from a full-body photo. It then generates optimal makeup, hairstyles, and outfits based on the situation and preferences. The generated suggestions are sent from the server to the device.

[1206] 4. Displaying the proposed results

[1207] The device analyzes the generated results received from the server and displays them in a format that is easy for the user to understand. The suggestions include the most suitable makeup, hairstyle, and outfit. The user checks the displayed suggestions and, if they like them, proceeds to the next step.

[1208] 5. Booking and payment processing

[1209] If the user likes the suggestions, they use the device's interface to search for nearby beauty salons, esthetic salons, and clothing stores, make a reservation, and pay. The device then sends the reservation information and payment data to the server, which relays it to the appropriate provider. The provider then sends a reservation confirmation to the server, which then processes the successful payment. Finally, the device displays a notification that the reservation and payment have been completed.

[1210] Specific examples

[1211] For example, consider the case where a user uploads a photo of their face and a full body in a situation where they want to know what kind of hairstyle is good for their date. If the user enters "I like casual styles," the system operates as follows:

[1212] 1. User input processing: The user uploads a photo of their face and a full-body photo, and enters the situation and preferences.

[1213] 2. Data transmission process: The device sends a face photo, a full-body photo, the situation, and preference information to the server.

[1214] 3. Data analysis and proposal generation process: The server analyzes the received information and proposes natural curly hair as a "casual hairstyle suitable for a date."

[1215] 4. Displaying the proposal results: The device displays the proposals for natural curly hair to the user.

[1216] 5. Booking and payment processing: The user decides to try the suggested hairstyle, searches for a nearby hair salon, makes a booking and makes payment.

[1217] In this way, users can easily find the makeup, hairstyle, and outfit that suits them, and can smoothly make reservations and make payments. This system provides optimal services to users who are interested in beauty and fashion but don't have much time.

[1218] The processing flow will be explained below.

[1219] Step 1:

[1220] The user launches the application. The device prompts the user for account authentication information, and the user enters the login information.

[1221] Step 2:

[1222] The user takes or uploads a photo of their face and a full-body photo, and then inputs the situation (e.g., date, work, party) and preferences (e.g., natural makeup, casual style).

[1223] Step 3:

[1224] The terminal transmits the face photo, the style photo, the situation, and the preference information to the server as a single data package. After the transmission is successful, the terminal notifies the user of the success.

[1225] Step 4:

[1226] The server inputs the received data into a generative AI model for analysis. The generative AI model extracts facial features (shape, skin tone, eyebrow shape, etc.) from the user's face photo and analyzes the user's body shape and style from a full-body photo.

[1227] Step 5:

[1228] The server generates the optimal makeup, hairstyle, and outfit based on the analysis results and the user's situation and preferences, and sends the generated results to the device.

[1229] Step 6:

[1230] The device analyzes the generated results received from the server and displays them in a format that is easy for the user to understand. The user checks the proposal and, if they like it, proceeds to the next step.

[1231] Step 7:

[1232] The user decides to make a reservation and make payment based on the proposed service. The terminal searches for nearby beauty salons, esthetic salons, and clothing stores and displays a list of businesses that can provide the proposed service.

[1233] Step 8:

[1234] The user selects a vendor and specifies a reservation date and time. After the selection and reservation information are confirmed, the terminal sends the reservation information and payment data to the server.

[1235] Step 9:

[1236] The server relays the reservation information to the merchant and receives confirmation of the reservation. The server processes the payment using PayPay and confirms the payment was successful.

[1237] Step 10:

[1238] The server sends a notification of completion of reservation and payment to the terminal. The terminal notifies the user that reservation and payment have been completed.

[1239] This series of processing flows allows users to easily and quickly receive suggestions for optimal makeup, hairstyles, and outfits, while also smoothly making reservations and making payments.

[1240] Example 1

[1241] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1242] Previously, to receive beauty and fashion recommendations, users had to search for multiple services and stores individually, make reservations, and pay. This was time-consuming and labor-intensive, making it difficult to receive optimal recommendations. Furthermore, technology to automatically generate recommendations tailored to a user's preferences and situation was insufficient, limiting the user experience.

[1243] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1244] In this invention, the server includes a means for inputting prompt text into the generative AI model to suggest makeup, hairstyles, and outfits, a means for the server to transmit the generated results to a terminal, and a means for the user to make reservations and payments based on the generated results. This allows users to receive optimal beauty and fashion suggestions simply by uploading photos of their face and style and inputting the situation and preferences. Furthermore, reservations and payments can be made all at once, significantly reducing the user's time and effort.

[1245] "User" refers to an individual or organization that uses the system.

[1246] "Terminal" refers to the device used by a User (e.g., smartphone, tablet, computer).

[1247] A "face photo" refers to an image of the user's face.

[1248] A "style photo" refers to an image of the user's entire body.

[1249] A "situation" refers to a specific event or purpose (e.g., date, work, party) entered by the user.

[1250] "Preferences" refers to the preferences regarding fashion and makeup input by the user (e.g., natural makeup, casual style).

[1251] "Server" refers to a computer system that processes, stores, and transmits data.

[1252] A "data package" refers to a collection of data including a face photo, style photo, situation, and preference information.

[1253] A "generative AI model" refers to an artificial intelligence algorithm that suggests makeup, hairstyles, and outfits based on input data.

[1254] A "prompt" refers to a specific instruction or request given to a generative AI model.

[1255] "Suggestion generation" refers to the process by which the generative AI model generates the best makeup, hairstyle, and outfit for the user based on the analysis results.

[1256] "Reservation" refers to the act of a user setting a date and time for using services at a beauty salon, esthetic salon, or clothing store based on the content suggested by the user.

[1257] "Payment" refers to the act of paying monetary consideration for the use of a service.

[1258] "Businesses" refer to service providers such as hair salons, beauty salons, and clothing stores.

[1259] "Generated results" refers to the makeup, hairstyle, and outfit suggestions generated by the generative AI model.

[1260] "Relay" refers to the act of the server forwarding information received from the user to the appropriate provider.

[1261] This invention is a system that allows users to receive suggestions for the best makeup, hairstyle, and outfits for themselves, and to make reservations and payments at beauty salons, esthetic salons, and clothing stores all in one place. This system is realized mainly using a terminal, a server, and a generative AI model.

[1262] Specific program description

[1263] User Input Processing

[1264] A user launches the application and uses their device to enter their account credentials to log in. After successfully logging in, the user uploads a face photo and a full-body photo to the device. In addition, the user inputs the specific situation (e.g., date, work, party) and their preferences (e.g., natural makeup, casual style), which allows the system to obtain the user's specific needs.

[1265] Data transmission process

[1266] The information entered by the user (face photo, full-body photo, situation, preferences) is sent as a single data package from the device to the server. The device notifies the user of the success of the data transmission.

[1267] Data analysis and proposal generation process

[1268] The server analyzes the received data package and inputs it as a prompt to the generative AI model. The generative AI model extracts facial features (shape, skin tone, eyebrow shape, etc.) from the user's face photo and analyzes body shape and style from a full-body photo. It also suggests the optimal makeup, hairstyle, and outfit based on the situation and preferences. The suggestions are sent from the server to the device.

[1269] Prompt Sentence Examples

[1270] "User uploaded a photo for a date. She prefers a casual look. Please suggest the best makeup and hairstyle for her."

[1271] Display of proposal results

[1272] The device analyzes the generated results received from the server and displays the proposals in a format that is easy for the user to understand. The user can review the proposals and, if they like them, proceed to the next step.

[1273] Reservation and payment processing

[1274] If the user likes the suggestions, they can search for nearby beauty salons, esthetic salons, and clothing stores via their device, make a reservation, and make payment. The reservation information and payment data are sent from the device to the server, which relays this information to the appropriate provider. After the server receives a reservation confirmation from the provider, the server processes the successful payment and displays a notification on the device that the reservation and payment have been completed.

[1275] Specific examples

[1276] For example, consider the case where a user uploads a photo of their face and a full body in a situation where they want to know what kind of hairstyle is good for their date. If the user enters "I like casual styles," the system operates as follows:

[1277] 1. User input processing: The user uploads a photo of their face and a full-body photo, and enters the situation and preferences.

[1278] 2. Data transmission process: The device sends a face photo, a full-body photo, the situation, and preference information to the server.

[1279] 3. Data analysis and proposal generation process: The server analyzes the received information and proposes natural curly hair as a "casual hairstyle suitable for a date."

[1280] 4. Displaying the proposal results: The device displays the proposals for natural curly hair to the user.

[1281] 5. Booking and payment processing: The user decides to try the suggested hairstyle, searches for a nearby hair salon, books a reservation and makes payment.

[1282] In this way, users can easily find the makeup, hairstyle, and outfit that suits them, and can smoothly make reservations and make payments. This system can provide an optimal service, especially for users who are interested in beauty and fashion but are short on time.

[1283] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1284] Specific explanation of processing steps

[1285] Step 1:

[1286] The user launches the application and goes to the login screen. The user enters their account information (username, password) and clicks the login button. The device sends this authentication information as input to the server. The server receives the authentication information and compares it with a database to determine whether the authentication was successful. If the authentication is successful, the server returns the information to the device, and the device outputs the home screen.

[1287] Step 2:

[1288] The user takes or selects a face photo and a full-body photo within the application and clicks the upload button. The device temporarily stores these photo data in memory. Next, a form is displayed in which the user can input the situation (e.g., date, work, party) and preferences (e.g., natural makeup, casual style). The user enters this information and clicks the submit button. These inputs are then compiled into a single data package.

[1289] Step 3:

[1290] The device sends the compiled data package (face photo, full-body photo, situation, and preference information) as input to the server. The server receives this data package and confirms its receipt. After confirmation, the server outputs a notification to the device that reception is complete. The device then notifies the user that the data transmission was successful.

[1291] Step 4:

[1292] The server analyzes the received data package. This analysis process uses facial recognition software (e.g., OpenCV) and data analysis software (e.g., TensorFlow). The server first extracts facial features (e.g., face shape, skin tone, eyebrow shape, etc.) from the face photo, and then analyzes body shape and style from the full-body photo. Next, it generates a prompt based on the situation and preferences. An example prompt is, "Please suggest makeup and hairstyle that suits this face photo and situation."

[1293] Step 5:

[1294] The server inputs the generated prompt into the generative AI model, which then suggests the best makeup, hairstyle, and outfit for the user based on the prompt. The generative AI model returns the generated suggestions (e.g., natural curly hair, beige trench coat, denim pants) to the server, which then receives them and sends them to the device.

[1295] Step 6:

[1296] The device analyzes the generated results received from the server and displays the suggestions in a user-friendly format. This allows the user to see specific makeup, hairstyle, and outfit suggestions. If the user is satisfied with the suggestions, they can proceed to the next step based on the suggestions.

[1297] Step 7:

[1298] If the user wants to try out the suggested service, they use the device's interface to search for nearby beauty salons, esthetic salons, and clothing stores. The device uses the user's current location information as input. The user selects the desired store from the displayed list and sets the reservation date and time. When the user clicks the reservation button, the device sends this reservation information to the server.

[1299] Step 8:

[1300] The server processes the received reservation information and relays it to the appropriate vendor. Once the server receives confirmation of the reservation from the vendor, the server initiates the payment process and notifies the terminal of successful payment. Finally, the terminal displays a notification to the user confirming the reservation and payment.

[1301] This allows users to easily find makeup, hairstyles, and outfits that suit them, and also makes reservations and payments smoothly.

[1302] (Application example 1)

[1303] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1304] In conventional beauty consulting and service booking systems, it was difficult for users to find the best makeup and hairstyle for themselves, and it was also difficult to smoothly book and pay with service providers, which caused many users to feel tedious and time-consuming.

[1305] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1306] In this invention, the server includes means for generating makeup, hairstyles, and outfits based on a user's facial photograph and style photograph using a generative AI model, means for transmitting the generated results to the terminal, and means for searching for and selecting a service provider for which a reservation has been made. This allows the user to easily find the makeup, hairstyle, and outfit that best suits them, and further allows for smooth booking and payment of related services all at once.

[1307] "User" refers to an individual who uses the system to upload a photo of their face and style, and input the situation and preferences.

[1308] "Mouthshot" refers to a photograph used to clearly show a user's facial features.

[1309] A "style photo" refers to a photo that shows the user's entire body and is used to show features such as body shape and clothing.

[1310] "Situation" refers to a specific scene or purpose for the makeup, hairstyle, and coordination desired by the user.

[1311] "Preferences" refer to the style and trends of makeup, hairstyle, and outfits desired by the user.

[1312] "Terminal" refers to an electronic device that a user uses to upload photos, input information, and communicate with a server.

[1313] "Server" refers to the central processing unit that receives and analyzes information sent by the user, generates optimal makeup, hairstyles, and outfits using a generative AI model, and sends the results to the terminal.

[1314] "Analysis" refers to the process by which the server analyzes the face photo, style photo, situation, and preference information it receives and makes appropriate suggestions based on that information.

[1315] A "generative AI model" refers to an artificial intelligence model that uses machine learning algorithms to analyze a user's characteristics and suggest optimal makeup, hairstyles, and outfits.

[1316] "Makeup, hairstyle, and coordination" refers to specific beauty and fashion suggestions that AI suggests based on the user's face photo, style photo, situation, and preferences.

[1317] "Generated results" refers to the makeup, hairstyle, and outfit suggestions generated by the generative AI model.

[1318] "Display" refers to the process by which the terminal visually shows the generated results to the user.

[1319] "Reservation" refers to a user making a reservation with a specific service provider based on the makeup, hairstyle, and outfit suggestions.

[1320] "Payment" refers to the process of paying for the service a user has reserved.

[1321] "Service provider" refers to a business that provides beauty and fashion-related services to users, such as a beauty salon, esthetic salon, or clothing store.

[1322] "Search" refers to the process performed by a terminal to find a particular service provider based on a user request.

[1323] "Selection" refers to the act of a user selecting a particular service provider from the search results.

[1324] System Overview

[1325] This invention is a system that allows users to receive recommendations for the best makeup, hairstyle, and outfits for themselves, and then make reservations and payments with on-site service providers all in one place. The system begins by users uploading photos of their face and style using a terminal, and inputting the situation and preferences. A specific embodiment of this system is described in detail below.

[1326] Program processing flow

[1327] User Input Processing

[1328] When a user launches the application, the device requests the user's account authentication information and prompts the user to log in. After logging in, the user uploads a face photo and a full-body photo, and enters the situation of the day (e.g., date, work, party) and their preferences (e.g., natural makeup, casual style).

[1329] Data transmission process

[1330] The device sends the facial photo, full-body photo, situation, and preference information acquired from the user to the server as a single data package. After the transmission is successful, the device notifies the user of the success.

[1331] Data analysis and proposal generation process

[1332] The server analyzes the received data and inputs it into a generative AI model. The generative AI model extracts facial features (shape, skin tone, eyebrow shape, etc.) from the user's face photo and analyzes the user's body type and style from a full-body photo. It then generates optimal makeup, hairstyles, and outfits based on the situation and preferences. The generated suggestions are sent from the server to the device.

[1333] Display of proposal results

[1334] The device analyzes the generated results received from the server and displays them in a format that is easy for the user to understand. The suggestions include the most suitable makeup, hairstyle, and outfit. The user checks the displayed suggestions and, if they like them, proceeds to the next step.

[1335] Reservation and payment processing

[1336] If the user likes the suggestions, they use the terminal interface to search for relevant service providers (e.g., hair salons, esthetic salons, clothing stores, etc.) and make a reservation and payment. The terminal sends the reservation information and payment data to the server, which relays it to the appropriate provider. The provider's reservation confirmation arrives at the server, which processes the successful payment. Finally, the terminal displays a notification that the reservation and payment have been completed.

[1337] Example of a system

[1338] When applying this system as a smartphone application, the following specific examples can be considered.

[1339] Examples:

[1340] For example, consider the case where a user uploads a photo of their face and a full body in a situation where they want to know what kind of hairstyle is good for their date. If the user enters "I like casual styles," the system operates as follows:

[1341] 1. The user uploads a photo of their face and a full-body photo, and enters the situation and preferences.

[1342] 2. The device sends a photo of the face, a full-body photo, the situation, and preference information to the server.

[1343] 3. The server analyzes the received information and uses a generative AI model to suggest natural curls as a "casual hairstyle suitable for a date."

[1344] 4. The device will display natural curl suggestions to the user.

[1345] 5. The user decides to try the suggested hairstyle and searches for a nearby hair salon, makes a reservation, and pays.

[1346] In this way, users can easily find the makeup, hairstyle, and outfit that suits them, and can also smoothly book and pay for related services.

[1347] Example prompts to input to the generative AI model:

[1348] "Please extract the user's features from face and full-body photos, and suggest the best makeup, hairstyle, and outfit based on the situation (date) and preference (casual)."

[1349] This system makes it possible to provide optimal services to users who are interested in beauty and fashion but don't have much time.

[1350] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1351] Step 1: User Input Processing

[1352] A user launches the application and enters their account authentication information on the login screen. After logging in, they upload a face photo and a full-body photo, and enter the situation of the day (e.g., date, work, party) and their preferences (e.g., natural makeup, casual style). This allows the user's characteristic information to be collected. The input data includes the face photo, full-body photo, situation, and preference information.

[1353] Step 2: Data transmission process

[1354] The device sends the face photo, full-body photo, situation, and preference information acquired from the user to the server as a single data package. The sent data consists of an image file and text information. When the server receives the data, it notifies the device of the success. The device then notifies the user of the success of the data transmission.

[1355] Step 3: Data analysis and proposal generation process

[1356] The server inputs the received data into a generative AI model for analysis. The generative AI model extracts facial features (e.g., facial contours, skin tone, eyebrow shape, etc.) from the face photo and analyzes body shape and clothing characteristics from the full-body photo. Next, it generates optimal makeup, hairstyle, and outfits based on the situation and preferences. The generated suggestions consist of specific advice on makeup, hairstyle, and outfits. Finally, these suggestions are sent from the server to the device.

[1357] Step 4: Displaying the proposed results

[1358] The terminal analyzes the generated results received from the server and displays them in a format that is easy for the user to understand. They are displayed in a specific screen layout (for example, images and descriptions of makeup, hairstyles, and outfits). The user checks these suggestions and, if they like them, proceeds to the next step. The input is the generated results from the server, and the output is the suggestions that the user visually checks.

[1359] Step 5: Booking and payment processing

[1360] If the user likes the proposed service, they use the terminal interface to search for nearby service providers (e.g., hair salons, esthetic salons, clothing stores, etc.). The user selects the desired service provider from the search results and makes a reservation and makes a payment. The terminal sends the reservation information and payment data as a single data package to the server. The server relays the received information and transmits the reservation information to the appropriate service provider. The server also receives reservation confirmation from the provider and notifies the user's terminal. The user can finally confirm that the reservation and payment have been completed. The input is the reservation and payment information, and the output is a notification that the reservation and payment have been completed.

[1361] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1362] System Overview

[1363] This invention is a system that allows users to receive recommendations for the best makeup, hairstyle, and outfits for themselves, and to make reservations and payments at beauty salons, esthetic salons, and clothing stores all in one place. Furthermore, this invention incorporates an emotion engine that analyzes the user's emotions, making it possible to make optimal recommendations based on the user's emotions. This system begins when the user uploads a photo of their face and style, and inputs the situation and preferences. Specific embodiments of this system are described in detail below.

[1364] Program processing flow

[1365] 1. User Input Processing

[1366] When a user launches the application, the device prompts the user for account authentication information, and the user enters the login information. After logging in, the user takes or uploads a face photo and a full-body photo. The user also enters the situation of the day (e.g., date, work, party) and their preferences (e.g., natural makeup, casual style).

[1367] 2. Emotion analysis processing

[1368] The device sends a facial photo to the emotion engine, which then analyzes the user's emotions from the facial photo. The emotion engine recognizes emotions such as happiness, surprise, sadness, and anger, and returns the analysis results to the device. The device then sends the emotion analysis results to the server.

[1369] 3. Data transmission process

[1370] The device sends the facial photo, full-body photo, situation, preference information, and emotion analysis results acquired from the user to the server as a single data package. After the transmission is successful, the device notifies the user of the success.

[1371] 4. Data analysis and proposal generation process

[1372] The server inputs the received data into a generative AI model for analysis. The generative AI model extracts facial features (shape, skin tone, eyebrow shape, etc.) from the user's face photo and analyzes the user's body type and style from a full-body photo. It then generates optimal makeup, hairstyles, and outfits based on the situation, preferences, and emotional analysis results. The generated suggestions are sent from the server to the device.

[1373] 5. Display of proposed results

[1374] The device analyzes the generated results received from the server and displays them in a format that is easy for the user to understand. The suggestions include the most suitable makeup, hairstyle, and outfit. The user reviews the suggestions and, if they like them, proceeds to the next step.

[1375] 6. Booking and payment processing

[1376] If the user likes the suggestions, they use the device's interface to search for nearby beauty salons, esthetic salons, and clothing stores, make a reservation, and pay. The device then sends the reservation information and payment data to the server, which relays it to the appropriate provider. The provider then sends a reservation confirmation to the server, which then processes the successful payment. Finally, the device displays a notification that the reservation and payment have been completed.

[1377] Specific examples

[1378] For example, consider a situation where a user uploads a photo of their face and a full body in a situation where they want to know what kind of hairstyle is best for their date. If the user enters "I like casual styles," the emotion engine analyzes the emotion of "happiness" from the user's face. The system works as follows:

[1379] 1. User input processing: The user uploads a photo of their face and a full-body photo, and enters the situation and preferences.

[1380] 2. Emotion analysis processing: The emotion engine analyzes the facial photo, recognizes the emotion of happiness, and sends the results back to the device.

[1381] 3. Data transmission process: The device sends a face photo, a full-body photo, the situation, preferences, and emotion analysis results to the server.

[1382] 4. Data analysis and proposal generation process: Based on the analysis results, the server proposes natural curly hair as a hairstyle suitable for a date that corresponds to casual and happy emotions.

[1383] 5. Displaying the proposal results: The device displays the proposals for natural curly hair to the user.

[1384] 6. Booking and payment processing: If the user likes the suggested hairstyle, they can search for nearby salons, make a booking and make a payment.

[1385] This series of processes allows users to easily find makeup, hairstyles, and outfits that suit them, and also allows for smooth booking and payment. By combining it with an emotion engine, more personalized suggestions can be made based on the user's emotions.

[1386] The processing flow will be explained below.

[1387] Step 1:

[1388] The user launches the application. The device prompts the user for account authentication information, and the user enters the login information.

[1389] Step 2:

[1390] The user takes or uploads a photo of their face and a full-body photo, and then inputs the situation (e.g., date, work, party) and preferences (e.g., natural makeup, casual style).

[1391] Step 3:

[1392] The device sends a facial photo to the emotion engine, which analyzes the user's facial photo and recognizes emotions such as happiness, surprise, sadness, and anger. The analysis results are then sent back to the device.

[1393] Step 4:

[1394] The terminal transmits the face photo, the style photo, the situation, the preference information, and the emotion analysis result to the server as a single data package. After the transmission is successful, the terminal notifies the user of the success.

[1395] Step 5:

[1396] The server inputs the received data into a generative AI model for analysis. The generative AI model extracts facial features (shape, skin tone, eyebrow shape, etc.) from the user's face photo and analyzes the user's body shape and style from a full-body photo.

[1397] Step 6:

[1398] The server generates the optimal makeup, hairstyle, and outfit based on the analysis results, situation, preferences, and emotion analysis results, and then sends the generated results to the device.

[1399] Step 7:

[1400] The device analyzes the generated results received from the server and displays them in an easy-to-understand format for the user. The user checks the suggestions and, if they like them, proceeds to the next step.

[1401] Step 8:

[1402] If the user likes the suggestions, they can use the device's interface to search for nearby beauty salons, esthetic salons, and clothing stores, and a list of businesses that can provide the suggested services will be displayed on the device.

[1403] Step 9:

[1404] The user selects a vendor and specifies the reservation date and time on a new reservation screen. After the selection and reservation information are confirmed, the terminal sends the reservation information and payment data to the server.

[1405] Step 10:

[1406] The server relays the reservation information to the vendor and receives confirmation of the reservation. The server processes the payment using PayPay and confirms the payment was successful.

[1407] Step 11:

[1408] The server sends a notification of completion of reservation and payment to the terminal. The terminal notifies the user that reservation and payment have been completed.

[1409] This detailed process flow allows users to easily find the makeup, hairstyle, and outfit that suits them, and also allows them to receive personalized suggestions based on emotion analysis. In addition, reservations and payments can be made smoothly, providing optimal service even to busy users.

[1410] Example 2

[1411] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1412] Conventional beauty recommendation systems have the problem that it takes a lot of time and effort for users to find the makeup, hairstyle, and coordination that is best suited to them. Furthermore, since the recommendations do not reflect the user's emotions or situation, the recommendations provided are difficult to personalize. Furthermore, users have to make separate reservations and payments at beauty salons, esthetic salons, and clothing stores based on the recommendations, making the process cumbersome.

[1413] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1414] In this invention, the server includes: means for a user to upload a face photo and a style photo; means for a user to input a situation and preferences; means for a terminal to transmit the face photo, the style photo, the situation, and the preference information to the server; means for the server to input the received information into a generative AI model, analyze facial features and style, and generate optimal makeup, hairstyle, and outfits; means for the terminal to transmit the face photo to an emotion analysis engine and analyze emotions; means for transmitting the emotion analysis results to the server; means for the server to generate a proposal using the emotion analysis results; means for the server to transmit the generation results to the terminal; means for the terminal to display the generation results to the user; means for the user to make a reservation and payment based on the generation results; and means for the server to process the reservation and payment. This allows a user to quickly receive optimal beauty suggestions personalized to them and further allows them to make a reservation and payment based on the proposals all at once.

[1415] "User" refers to any individual or organization that uses this system.

[1416] "Terminal" refers to an electronic device (e.g., a smartphone or PC) operated by a user.

[1417] A "server" refers to a computer that manages the entire system, processes and stores data, and runs generative AI models.

[1418] "Face photo" refers to an image file of a user's face.

[1419] A "style photo" refers to an image file that captures the user's entire body.

[1420] "Situation" refers to the situation or event of the day (e.g., date, work, party) entered by the user.

[1421] "Preferences" refers to personal style and makeup preferences entered by the user (e.g., natural makeup, casual style).

[1422] A "generative AI model" refers to an artificial intelligence model that generates optimal makeup, hairstyles, and outfits based on input data.

[1423] An "emotion analysis engine" refers to software that analyzes emotions from a user's facial photo and provides the results.

[1424] "Data package" refers to a data set that compiles user-entered facial photos, style photos, situations, preference information, and emotion analysis results.

[1425] A "reservation" refers to a user setting a time and date at a store such as a beauty salon, esthetic salon, or clothing store to receive the proposed beauty service.

[1426] "Payment" means the online payment of the service booked by the User.

[1427] MODE FOR CARRYING OUT THE INVENTION

[1428] This invention is a system that allows users to receive recommendations for the best makeup, hairstyle, and outfits for themselves, and to make reservations and payments at hair salons, esthetic salons, and clothing stores all in one place. Furthermore, by incorporating an emotion analysis engine, it is possible to make optimal recommendations based on the user's emotions. This system mainly uses the following hardware and software:

[1429] Hardware and software used

[1430] Device: Electronic device operated by the user, such as a smartphone or computer

[1431] Server: A computer that manages the entire system, processes and stores data, and runs generative AI models.

[1432] Generative AI model: An artificial intelligence model that generates optimal makeup, hairstyles, and outfits based on user information (e.g., "StyleAI")

[1433] Emotion analysis engine: Software that analyzes emotions from a user's facial photo and provides the results (e.g., "EmotionAI")

[1434] Specific examples of programs

[1435] The system starts by having users upload a photo of their face and style, and input the situation and preferences.

[1436] 1. User Input Processing

[1437] When a user launches the application, the device prompts the user for account authentication information, and the user enters the login information. After successful login, the user takes or uploads a face photo and a full-body photo, and enters the situation (e.g., date, work, party) and preferences (e.g., natural makeup, casual style).

[1438] 2. Emotion analysis processing

[1439] The device sends a facial photo to the emotion analysis engine, which then analyzes the user's emotions from the facial photo. The emotion analysis engine recognizes emotions such as happiness, surprise, sadness, and anger, and returns the analysis results to the device. The device then sends the emotion analysis results to the server.

[1440] 3. Data transmission process

[1441] The device sends the facial photo, full-body photo, situation, preference information, and emotion analysis results acquired from the user to the server as a single data package. After the transmission is successful, the device notifies the user of the success.

[1442] 4. Data analysis and proposal generation process

[1443] The server inputs the received data into a generative AI model. The generative AI model extracts facial features (shape, skin tone, eyebrow shape, etc.) from the user's face photo and analyzes the user's body type and style from a full-body photo. It then generates optimal makeup, hairstyles, and outfits based on the situation, preferences, and emotional analysis results. The generated suggestions are sent from the server to the device.

[1444] 5. Display of proposed results

[1445] The device analyzes the generated results received from the server and displays them in a format that is easy for the user to understand. The suggestions include the most suitable makeup, hairstyle, and outfit. The user reviews the suggestions and, if they like them, proceeds to the next step.

[1446] 6. Booking and payment processing

[1447] If the user likes the suggestions, they use the device's interface to search for nearby beauty salons, esthetic salons, and clothing stores, make a reservation, and pay. The device then sends the reservation information and payment data to the server, which relays it to the appropriate provider. The provider then sends a reservation confirmation to the server, which then processes the successful payment. Finally, the device displays a notification that the reservation and payment have been completed.

[1448] Specific examples

[1449] For example, if a user uploads a photo of their face and a full body photo in a situation where they have a date and want to know what kind of hairstyle is nice, the following will happen.

[1450] Users upload photos of their face and body, and enter the situation and preferences.

[1451] The emotion analysis engine analyzes the facial photo, recognizes the emotion of happiness, and sends the results back to the device, which then sends the emotion analysis results to the server.

[1452] The device sends a photo of the face, a full-body photo, the situation, preferences, and emotional analysis results to the server.

[1453] The server inputs the data into a generative AI model, which suggests natural curls as a hairstyle suitable for a date, corresponding to casual and happy feelings.

[1454] The device displays natural curl suggestions to the user.

[1455] The user likes the suggested hairstyle, searches for a nearby hair salon, makes a reservation, and makes payment.

[1456] This series of processes allows users to easily find the makeup, hairstyle, and outfit that suits them, and also allows for smooth booking and payment. By combining it with a sentiment analysis engine, more personalized suggestions can be made based on the user's emotions.

[1457] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1458] Program processing flow

[1459] Step 1: User Input Processing

[1460] Input: User login information, face photo, full-body photo, situation, preferences

[1461] Output: Face photo, full body photo, situation, preference data package

[1462] Specific behavior:

[1463] 1. The user launches the application and enters login information (username and password) into the terminal.

[1464] 2. The device sends the login information to the server for authentication. If the login is successful, the device displays a screen for the user to take or upload a face photo and a full-body photo.

[1465] 3. The user uploads a face photo and a full-body photo. The device then displays a situation input screen, and the user selects an appropriate situation (e.g., date, work, party).

[1466] 4. Next, the device displays a preference input screen, and the user selects their preference (e.g., natural makeup, casual style).

[1467] Step 2: Sentiment Analysis Processing

[1468] Input: Face photo

[1469] Output: Emotion analysis results

[1470] Specific behavior:

[1471] 1. The device sends a photo of the user's face to an emotion analysis engine (e.g., "EmotionAI").

[1472] 2. The emotion analysis engine analyzes the facial photo, recognizes emotions such as happiness, surprise, sadness, and anger, and sends the results back to the device.

[1473] 3. The device receives the emotion analysis results and sends them to the server.

[1474] Step 3: Data transmission process

[1475] Input: Face photo, full-body photo, situation, preferences, emotion analysis results

[1476] Output: Send data package to server

[1477] Specific behavior:

[1478] 1. The device compiles the facial photo, full-body photo, situation, preference information, and emotion analysis results obtained from the user into a single data package.

[1479] 2. The device sends this data package to the server, and after successful transmission, the device notifies the user of the success.

[1480] Step 4: Data analysis and proposal generation process

[1481] Input: Data package (face photo, full-body photo, situation, preferences, emotion analysis results)

[1482] Output: Suggestions (optimal makeup, hairstyle, outfit)

[1483] Specific behavior:

[1484] 1. The server inputs the received data package into a generative AI model (e.g., "StyleAI") for analysis.

[1485] 2. The generative AI model extracts facial features (shape, skin tone, eyebrow shape, etc.) from a photo of the user's face and analyzes the user's body shape and style from a full-body photo.

[1486] 3. The server generates the optimal makeup, hairstyle, and outfit based on the situation, preferences, and emotional analysis results.

[1487] 4. The generated proposal is sent from the server to the device.

[1488] Step 5: Displaying the proposed results

[1489] Input: Suggestion details (best makeup, hairstyle, outfit)

[1490] Output: Suggestion results to be displayed

[1491] Specific behavior:

[1492] 1. The terminal analyzes the generated results received from the server and displays them in a format that is easy for the user to understand.

[1493] 2. The suggestions include the best makeup, hairstyle, and outfit.

[1494] 3. The user reviews the suggestions and, if they like them, proceeds to the next step.

[1495] Step 6: Booking and payment processing

[1496] Input: Proposal details, reservation information, payment information

[1497] Output: Notification of reservation and payment completion

[1498] Specific behavior:

[1499] 1. If the user likes the suggestions, the device provides an interface for searching for nearby hair salons, beauty salons, and clothing stores.

[1500] 2. The user selects the desired store and enters reservation information (date, time, service details, etc.).

[1501] 3. The terminal prompts the user to enter payment information (such as credit card information), after which the terminal sends the reservation information and payment data to the server.

[1502] 4. The server relays the reservation information and payment data to the appropriate vendor, and the vendor sends a confirmation of the reservation to the server.

[1503] 5. The server processes the successful payment and finally displays a notification on the terminal that the reservation and payment are complete.

[1504] (Application example 2)

[1505] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1506] While conventional beauty recommendation systems can suggest optimal makeup, hairstyles, and outfits to users, they face the challenge of making it difficult to actually try out and confirm the suggestions. Furthermore, when making reservations and paying at beauty salons or clothing stores based on the suggestions, multiple steps are required, which is cumbersome for users. This results in limited accuracy of the suggestions and a poor user experience.

[1507] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for generating makeup, hairstyles, and outfits based on the user's face photo, style photo, situation, and preferences using a generative AI model, means for transmitting the generated results to the terminal, and means for the terminal to have the user try on the generated results using a virtual reality device. This allows the user to intuitively check the proposed styles in real time, and also enables reservations and payments to be made all at once.

[1508] "User" refers to an individual who uses this system.

[1509] "Facial photo" refers to image data of the user's face.

[1510] A "style photo" refers to image data that shows the user's entire body.

[1511] "Situation" refers to the purpose or situation in which a user uses the system.

[1512] "Preferences" refer to the fashion and makeup styles desired by the user.

[1513] "Terminal" refers to a device used to use this system, such as a smartphone, tablet, or PC.

[1514] "Server" refers to a remote computer system that communicates with and processes data from a terminal.

[1515] A "generative AI model" refers to an artificial intelligence algorithm that suggests optimal makeup, hairstyles, and outfits based on a user's face photo, style photo, situation, and preferences.

[1516] "Generated results" refers to the makeup, hairstyle, and outfit suggestions obtained using the generative AI model.

[1517] "Means for displaying" refers to the function of visually showing the generated results on the terminal screen.

[1518] "Reservation" refers to the act of a user reserving services in advance at a beauty salon, clothing store, or the like in order to actually perform the suggested makeup or hairstyle.

[1519] "Payment" refers to the act of paying in advance for the service a user uses.

[1520] "Virtual reality device" refers to equipment (e.g., a head-mounted display or smart glasses) that allows users to try out generated results in a virtual space.

[1521] "Trying on" refers to a user virtually experiencing suggested makeup, hairstyles, and outfits using a virtual reality device.

[1522] This invention is a system that allows users to receive personalized recommendations for their best makeup, hairstyle, and outfits, and to make reservations and payments at beauty salons and clothing stores all in one place. The system combines an emotion analysis engine with a generative AI model and uses virtual reality devices to enhance the user experience.

[1523] System Overview

[1524] The server receives the face and style photos uploaded by the user, as well as the situation and preference information, and uses a generative AI model to generate the optimal makeup, hairstyle, and outfit.The generated results are then sent to the device and displayed to the user.

[1525] The terminal sends a photo of the user's face and style to the server in response to user operation, receives the generated results, and displays them. The user can also try out the generated results using a virtual reality device. Furthermore, the user can make reservations and make payments at nearby beauty salons or clothing stores based on the generated results they like.

[1526] Hardware and software used

[1527] Hardware:

[1528] Devices such as smartphones, tablets, and computers

[1529] Virtual reality devices (e.g., head-mounted displays, smart glasses)

[1530] software:

[1531] Emotion Analysis Engine: An algorithm for analyzing emotions from user facial photos

[1532] Generative AI model: An artificial intelligence that generates optimal makeup, hairstyles, and outfits based on information submitted by the user.

[1533] API: Manages the sending and receiving of data between the device and the server

[1534] Data processing and calculation

[1535] The server processes the following data:

[1536] Receives a photo of your face and style, and uses image analysis algorithms to extract facial features (shape, skin tone, eyebrow shape, etc.) and body shape

[1537] The situation (date, work, party, etc.), preferences (natural makeup, casual style, etc.) and emotion analysis results (happiness, surprise, sadness, anger, etc.) are taken into account and input into the generative AI model.

[1538] The generative AI model generates optimal makeup, hairstyle, and outfit suggestions, and sends the results to the device.

[1539] Specific examples

[1540] For example, consider a situation where a user uploads a photo of their face and a full body in a situation where they want to know what kind of hairstyle is best for their date. If the user enters "I like casual styles," and the emotion engine analyzes the emotion "happy" from the user's face, the server will perform the following steps:

[1541] Using a generative AI model based on the situation, preferences, and emotional analysis, the app suggests hairstyles suitable for casual dates.

[1542] The proposed natural curls are sent to the device, which then displays them to the user.

[1543] The device uses a virtual reality device to allow the user to try on suggested hairstyles.

[1544] If the user likes the suggested hairstyle, they can search for nearby hair salons, make a reservation, and pay in one go.

[1545] Prompt Sentence Examples

[1546] An example of a prompt for a generative AI model is:

[1547] "Please suggest the best hairstyle based on the user's face photo, full-body photo, situation (date), preference (casual style), and emotion (happiness)."

[1548] In this way, users can easily find the makeup, hairstyle, and outfit that best suits them, and can check it in real time in the VR space, making reservations and payments smoothly.

[1549] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1550] Step 1:

[1551] The user takes or uploads a photo of their face and whole body, and inputs the situation (e.g., date, work, party) and preferences (e.g., natural makeup, casual style). This information is then added to the device as input data.

[1552] Step 2:

[1553] The device sends a facial photo to an emotion analysis engine, which analyzes the user's emotions (e.g., happiness, surprise, sadness, anger). Specifically, a facial recognition algorithm is used to extract emotions from the image and provide the analysis results.

[1554] Step 3:

[1555] The emotion analysis engine returns the emotion analysis results to the terminal, which then sends these analysis results along with other input data (face photo, full-body photo, situation, preferences, etc.) to the server as a packet.

[1556] Step 4:

[1557] The server analyzes the received data packets and inputs them into a generative AI model. The generative AI model extracts facial features (shape, skin tone, eyebrow shape, etc.) from the user's face photo and analyzes the user's body type and style from a full-body photo. It then generates optimal makeup, hairstyles, and outfits based on the situation, preferences, and emotional analysis results.

[1558] Step 5:

[1559] The server then sends the generated makeup, hairstyle, and outfit suggestions to the device. The generated suggestions are formatted in a way that is intuitively understandable to the user.

[1560] Step 6:

[1561] The device then transmits the generated results to a virtual reality device, which can be a head-mounted display or smart glasses, allowing the user to try out the proposed styles in real time in a virtual space.

[1562] Step 7:

[1563] If the user tries out the suggested makeup, hairstyle, and outfit and likes it, they can search for nearby beauty salons and clothing stores through the device, make a reservation, and pay. The device presents these operations to the user as a series of processes.

[1564] Step 8:

[1565] The terminal sends the user's reservation and payment information to the server, which then relays it to the corresponding vendor. Once the server receives the reservation confirmation from the vendor, the server completes the payment process, sends the final confirmation information to the terminal, and notifies the user.

[1566] This series of processes allows users to easily find the makeup, hairstyle, and outfit that best suits them, and they can then check it in real time in the VR space, making reservations and payments smoothly.

[1567] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the control target 443 to output the result of the specific processing. The microphone 238 acquires voice indicating a user input regarding the result of the specific processing. The control unit 46A transmits voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[1568] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1569] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the robot 414.

[1570] The emotion identification model 59 as an emotion engine may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to an emotion map (see FIG. 9), which is a specific mapping. Similarly, the emotion identification model 59 may determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[1571] FIG. 9 is a diagram illustrating an emotion map 400 on which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. Emotions closer to the center of the concentric circles are more primitive. Emotions representing states and actions arising from a state of mind are arranged on the outer edges of the concentric circles. The concept of emotion includes both affect and mental states. Emotions generally generated from reactions occurring in the brain are arranged on the left side of the concentric circles. Emotions generally induced by situational judgment are arranged on the right side of the concentric circles. Emotions generally generated from reactions occurring in the brain and induced by situational judgment are arranged on the upper and lower sides of the concentric circles. Furthermore, the emotion of "pleasure" is arranged on the upper side of the concentric circles, and the emotion of "discomfort" is arranged on the lower side. In this way, in the emotion map 400, multiple emotions are mapped based on the structure by which emotions are generated, and emotions that tend to occur simultaneously are mapped close to each other.

[1572] These emotions are distributed in the 3 o'clock direction on emotion map 400, and typically fluctuate between relief and anxiety. In the right half of emotion map 400, situational awareness dominates over internal sensations, resulting in a sense of calm.

[1573] The inside of emotion map 400 represents what is going on in the mind, and the outside of emotion map 400 represents behavior, so the further you go outside emotion map 400, the more visible the emotions become (the more they are expressed in behavior).

[1574] Human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. Emotions can also be created for robots, automobiles, and motorcycles, based on various balances, such as posture and remaining battery life. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. An emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on Voice Emotion Recognition and Emotional Brain Physiological Signal Analysis Systems, Tokushima University, Doctoral Dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map lists emotions belonging to the "reaction" domain, where sensation is dominant. The right half of the emotion map lists emotions belonging to the "situation" domain, where situational awareness is dominant.

[1575] The emotion map defines two emotions that promote learning. One is a negative emotion on the situation side, around the middle of "repentance" or "reflection." In other words, this occurs when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is a positive emotion on the response side, around "desire." In other words, this occurs when the robot experiences positive feelings such as "I want more" or "I want to know more."

[1576] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​indicating each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple pieces of training data that are combinations of user input and emotion values ​​indicating each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions that are located close to each other have similar values, as in the emotion map 900 shown in FIG. 10. FIG. 10 shows an example in which multiple emotions, "relieved," "calm," and "reassuring," have similar emotion values.

[1577] The system according to the present disclosure has been described above mainly with respect to the functions of the data processing device 12, but the system according to the present disclosure is not necessarily implemented on a server. The system according to the present disclosure may be implemented as a general information processing system. The present disclosure may be implemented, for example, as a software program running on a personal computer or an application running on a smartphone, etc. The method according to the present disclosure may be provided to users in the form of SaaS (Software as a Service).

[1578] In the above embodiment, an example was given in which the specific processing is performed by one computer 22, but the technology of the present disclosure is not limited to this, and the specific processing may be distributed and performed by a plurality of computers including the computer 22. For example, the data generation model 58 may be provided in an external device of the data processing device 12, and data may be generated in the external device in accordance with input data.

[1579] In the above embodiment, an example in which the specific processing program 56 is stored in the storage 32 has been described, but the technology of the present disclosure is not limited to this. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-transitory storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-transitory storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes the specific processing in accordance with the specific processing program 56.

[1580] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[1581] It is not necessary to store all of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store all of the specific processing program 56 in the storage 32; only a portion of the specific processing program 56 may be stored.

[1582] The hardware resource for executing a specific process can be any of the following processors: An example of a processor is a CPU, which is a general-purpose processor that functions as a hardware resource for executing a specific process by executing software, i.e., a program. Another example of a processor is a dedicated electrical circuit, such as an FPGA (Field-Programmable Gate Array), a PLD (Programmable Logic Device), or an ASIC (Application Specific Integrated Circuit), which is a processor with a circuit configuration designed specifically for executing a specific process. Each processor has built-in or connected memory, and each processor uses the memory to execute the specific process.

[1583] The hardware resource that executes the specific processing may be configured with one of these various processors, or may be configured with a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Also, the hardware resource that executes the specific processing may be a single processor.

[1584] As an example of a system configured with a single processor, first, one processor is configured by combining one or more CPUs and software, and this processor functions as a hardware resource that executes a specific process. Second, there is a system that uses a processor that realizes the functions of an entire system including multiple hardware resources that execute a specific process on a single IC chip, as typified by SoC (System-on-a-chip). In this way, a specific process is realized using one or more of the above-mentioned various processors as hardware resources.

[1585] Furthermore, the hardware structure of these various processors can be, more specifically, an electric circuit that combines circuit elements such as semiconductor devices. The specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps may be deleted, new steps may be added, or the processing order may be rearranged, without departing from the spirit of the invention.

[1586] The above-described description and illustrations are a detailed explanation of the parts related to the technology of the present disclosure and are merely an example of the technology of the present disclosure. For example, the above description of the configuration, functions, actions, and effects is an explanation of an example of the configuration, functions, actions, and effects of the parts related to the technology of the present disclosure. Therefore, it goes without saying that unnecessary parts may be deleted, new elements may be added, or replacements may be made to the above-described description and illustrations within the scope of the gist of the technology of the present disclosure. Furthermore, to avoid confusion and facilitate understanding of the parts related to the technology of the present disclosure, the above-described description and illustrations omit explanations of common technical knowledge that do not require particular explanation to enable the implementation of the technology of the present disclosure.

[1587] All publications, patent applications, and technical standards mentioned in this specification are herein incorporated by reference to the same extent as if each individual publication, patent application, or technical standard was specifically and individually indicated to be incorporated by reference.

[1588] The following is further disclosed regarding the above embodiment.

[1589] (Claim 1)

[1590] A means for users to upload photos of their face and style;

[1591] a means for the user to input situations and preferences;

[1592] a means for transmitting the face photo, the style photo, the situation, and the preference information to a server by a terminal;

[1593] A means for the server to analyze the received information and generate makeup, hairstyles, and outfits using a generative AI model;

[1594] A means for the server to transmit the generated result to a terminal;

[1595] a means for displaying the generated result to a user in a terminal;

[1596] A means for a user to make a reservation and payment based on the generated results;

[1597] means for a server to process said reservations and payments;

[1598] A system including:

[1599] (Claim 2)

[1600] 2. The system according to claim 1, further comprising means for analyzing the generated results received by the terminal and displaying them in a user-friendly format.

[1601] (Claim 3)

[1602] 2. The system of claim 1, further comprising means for the server to relay reservations and payments made by the user to the merchant and to receive confirmation.

[1603] "Example 1"

[1604] (Claim 1)

[1605] A means for users to upload photos of their face and style;

[1606] a means for the user to input situations and preferences;

[1607] a means for transmitting the face photo, the style photo, the situation, and the preference information to a server by a terminal;

[1608] A means for the server to analyze the received information and generate makeup, hairstyles, and coordination using a generation AI model;

[1609] A method for suggesting makeup, hairstyles, and outfits by inputting prompt sentences into the generative AI model, and

[1610] A means for the server to transmit the generated result to a terminal;

[1611] a means for displaying the generated result to a user in a terminal;

[1612] A means for a user to make a reservation and payment based on the generated results;

[1613] means for a server to process said reservations and payments;

[1614] A system including:

[1615] (Claim 2)

[1616] 2. The system according to claim 1, further comprising means for analyzing the generated results received by the terminal and displaying them in a user-friendly format.

[1617] (Claim 3)

[1618] 2. The system of claim 1, further comprising means for the server to relay reservations and payments made by the user to the merchant and to receive confirmation.

[1619] "Application Example 1"

[1620] (Claim 1)

[1621] A means for users to upload photos of their face and style;

[1622] a means for the user to input situations and preferences;

[1623] a means for transmitting the face photo, the style photo, the situation, and the preference information to a server by a terminal;

[1624] A means for the server to analyze the received information and generate makeup, hairstyles, and outfits using a generative AI model;

[1625] A means for the server to transmit the generated result to a terminal;

[1626] a means for displaying the generated result to a user in a terminal;

[1627] A means for a user to make a reservation and payment based on the generated results;

[1628] means for a server to process said reservations and payments;

[1629] means for searching and selecting a service provider for which a reservation has been made;

[1630] A system including:

[1631] (Claim 2)

[1632] 2. The system according to claim 1, further comprising means for analyzing the generated results received by the terminal and displaying them in a user-friendly format.

[1633] (Claim 3)

[1634] 2. The system of claim 1, further comprising means for the server to relay reservations and payments made by the user to the merchant and to receive confirmation.

[1635] "Example 2: Combining Emotion Engines"

[1636] (Claim 1)

[1637] A means for users to upload photos of their face and style;

[1638] a means for the user to input situations and preferences;

[1639] a means for transmitting the face photo, the style photo, the situation, and the preference information to a server by a terminal;

[1640] The server inputs the received information into a generation AI model, analyzes facial features and style, and generates optimal makeup, hairstyles, and outfits.

[1641] A means for the terminal to send a facial photo to an emotion analysis engine and analyze emotions;

[1642] means for transmitting the emotion analysis result to a server;

[1643] a server generating a suggestion using the sentiment analysis results;

[1644] A means for the server to transmit the generated result to a terminal;

[1645] a means for displaying the generated result to a user in a terminal;

[1646] A means for a user to make a reservation and payment based on the generated results;

[1647] means for a server to process said reservations and payments;

[1648] A system including:

[1649] (Claim 2)

[1650] 2. The system according to claim 1, further comprising means for analyzing the generated results received by the terminal and displaying them in a user-friendly format.

[1651] (Claim 3)

[1652] 2. The system of claim 1, further comprising means for the server to relay reservations and payments made by the user to the merchant and to receive confirmation.

[1653] "Application example 2 when combining emotion engines"

[1654] (Claim 1)

[1655] A means for users to upload photos of their face and style;

[1656] a means for the user to input situations and preferences;

[1657] a means for transmitting the face photo, the style photo, the situation, and the preference information to a server by a terminal;

[1658] A means for the server to analyze the received information and generate makeup, hairstyles, and outfits using a generative AI model;

[1659] A means for the server to transmit the generated result to a terminal;

[1660] a means for displaying the generated result to a user in a terminal;

[1661] A means for a user to make a reservation and payment based on the generated results;

[1662] means for a server to process said reservations and payments;

[1663] A means for the terminal to try on the generated result using a virtual reality device;

[1664] A system including:

[1665] (Claim 2)

[1666] 2. The system according to claim 1, further comprising means for analyzing the generated results received by the terminal and displaying them in a user-friendly format.

[1667] (Claim 3)

[1668] 2. The system of claim 1, further comprising means for the server to relay reservations and payments made by the user to the merchant and to receive confirmation. [Explanation of symbols]

[1669] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Device 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robot< / url:> < / url:> < / url:> < / url:>

Claims

1. A means for users to upload photos of their face and style; a means for the user to input situations and preferences; a means for transmitting the face photo, the style photo, the situation, and the preference information to a server by a terminal; A means for the server to analyze the received information and generate makeup, hairstyles, and outfits using a generative AI model; A means for the server to transmit the generated result to a terminal; a means for displaying the generated result to a user in a terminal; A means for a user to make a reservation and payment based on the generated results; means for a server to process said reservations and payments; A system including:

2. 2. The system according to claim 1, further comprising means for analyzing the generated results received by the terminal and displaying them in a user-friendly format.

3. 2. The system of claim 1, further comprising means for the server to relay reservations and payments made by the user to the merchant and to receive confirmation.

Images