System

Abstract

A system is provided.SOLUTION: A system, comprising: means for a user to upload a face picture; means for sending the face picture to a server; means for the server to use a generative AI model to analyze the face picture; means for suggesting makeup, hairstyle, and fashion that suits the user's preferences based on an analytical result obtained by the generative AI model; means for making a reservation at a beauty salon, an aesthetic salon, or a fashion store based on the suggested makeup, hairstyle, and fashion; and means for making a payment through a settlement platform.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] Today, interest in beauty and fashion is on the rise, with women in their 20s to 40s in particular placing a strong emphasis on self-expression. However, many people find it difficult to find a style or outfit that suits them. Furthermore, making reservations and paying at beauty salons, esthetic salons, and fashion stores can be a time-consuming process, placing a significant burden on busy people. Therefore, there is a need for a system that allows users to receive personalized beauty and fashion recommendations and smoothly make reservations and payments. [Means for solving the problem]

[0005] The present invention provides a system that includes a means for a user to upload a facial photo, a means for transmitting the facial photo to a server, a means for the server to use a generative AI model that analyzes the facial photo, a means for suggesting makeup, hairstyles, and fashions that match the user's preferences based on the analysis results obtained by the generative AI model, a means for making reservations at beauty salons, esthetic salons, and fashion stores based on the suggested makeup, hairstyles, and fashions, and a means for making payments through a payment platform.The system analyzes the user's beauty and fashion information and makes optimal suggestions, thereby improving the user experience and centralizing and simplifying the complicated reservation and payment procedures.Furthermore, the system includes a means for inputting user preferences and trend information and having the generative AI model make optimal suggestions based thereon, and a means for the server to notify the beauty salon, esthetic salon, and fashion store of reservation information and receive reservation confirmation information, thereby enhancing user convenience and realizing a new form of service in the beauty and fashion market.

[0006] "User" refers to an individual who uses this system, in particular, a person who wishes to upload a photo of their face and receive suggestions on makeup, hairstyles, and fashion.

[0007] A "face photo" refers to a photograph showing a user's entire face, and refers to image data used to analyze facial features, contours, and skin tone.

[0008] "Server" refers to the computer system that receives the user's facial photo and input information, and analyzes and makes suggestions using a generative AI model.

[0009] A "generative AI model" is an algorithm that uses machine learning technology to analyze a user's facial photo, preferences, and trend information, and suggests the best makeup, hairstyle, and fashion for the user.

[0010] "Makeup" refers to suggested makeup and cosmetic application methods based on a facial photo.

[0011] "Hairstyle" refers to the hairstyle or hair arrangement style suggested based on a facial photo.

[0012] "Fashion" refers to styling of clothing, accessories, etc. suggested based on the user's photo and preferences.

[0013] "Reservation" refers to the procedure by which a user reserves services at a beauty salon, esthetic salon, fashion store, etc. in advance based on a suggestion.

[0014] "Payment" refers to the act of paying for reserved services at a hair salon, beauty salon, or fashion store via a payment platform.

[0015] "Payment Platform" means the online or mobile payment system used by Users to pay for the Services. [Brief explanation of the drawings]

[0016] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8]FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 3 is a sequence diagram showing a processing flow of the data processing system according to the first embodiment. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0017] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0018] First, the terms used in the following description will be explained.

[0019] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0020] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0021] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0022] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0023] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0024] [First embodiment]

[0025] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0026] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0027] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0028] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0029] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0030] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0031] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0032] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0033] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0034] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0035] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0036] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0037] The present invention is a system that allows users to receive personalized beauty and fashion recommendations and smoothly make reservations and payments. The following elements are required to implement this system:

[0038] A way for users to upload photos of their faces

[0039] User:

[0040] 1. Launch the application and take a photo of your face using the camera function or select an existing photo from your gallery.

[0041] 2. Upload the facial photo data to the application through the interface for uploading facial photos.

[0042] A means of sending a face photo to the server

[0043] Device:

[0044] 1. The uploaded facial photo data is temporarily saved and prepared for sending to the server.

[0045] 2. The saved facial photo data is sent to the server as an HTTP request.

[0046] The server generates a facial photo and analyzes it using an AI model.

[0047] server:

[0048] 1. The facial photo data received from the device is input into the generative AI model.

[0049] 2. The generative AI model analyzes the facial features (e.g., facial contours, skin tone, eye shape, etc.) and converts them into numerical data.

[0050] 3. Store the analysis results in an internal database and forward them for further processing.

[0051] How generative AI models generate proposals

[0052] server:

[0053] 1. Obtain the analysis results of the user's facial photo from the internal database.

[0054] 2. User-input preferences and current trend information are fed into the AI ​​model to generate optimal makeup, hairstyle, and fashion suggestions.

[0055] 3. The generated proposal results are sent to the device in JSON format.

[0056] A way to make a reservation based on the suggestions

[0057] User:

[0058] 1. Check the suggestions and select your favorite hair salon, beauty salon, or fashion store.

[0059] 2. Enter reservation information such as the service content, date and time of the selected store and confirm your reservation.

[0060] A means of making payments through a payment platform

[0061] Device:

[0062] 1. The reservation information entered by the user is sent to the server.

[0063] 2. An interface for entering payment information is displayed, and the user enters the payment information.

[0064] 3. The entered payment information is sent to the payment platform and the processing result is received.

[0065] server:

[0066] 1. Notify beauty salons, esthetic salons, and fashion stores of reservation information received from the device.

[0067] 2. Receive a payment confirmation from the payment platform to confirm the completion of your booking.

[0068] 3. Send the completed booking and payment information to the terminal.

[0069] Specific examples

[0070] Example 1: Makeup and hairstyle suggestions

[0071] User:

[0072] 1. Upload a photo of your face using the application and indicate that you are interested in natural makeup and bob hairstyles.

[0073] 2. Check the suggestions from the server and make a reservation at the suggested beauty salon.

[0074] Device:

[0075] 1. Send a face photo and user preferences to the server.

[0076] 2. Receive suggestions from the server and present them to the user.

[0077] 3. Send the reservation information to the server and display the payment interface.

[0078] server:

[0079] 1. Analyze your facial photo and suggest the best makeup and hairstyle.

[0080] 2. Notify the beauty salon of the user's reservation information and receive payment completion information.

[0081] Example 2: Fashion coordination suggestions

[0082] User:

[0083] 1. Enter your interest in casual style and upload a photo of your face.

[0084] 2. Make an appointment at the suggested fashion store and complete the payment online.

[0085] Device:

[0086] 1. Send your face photo and preference information to the server.

[0087] 2. Receive suggestions from the server and present them to the user.

[0088] 3. Send the reservation information to the server and process the payment.

[0089] server:

[0090] 1. We suggest the best fashion coordination based on the analysis of your face photo and your preferences.

[0091] 2. The user's reservation information is notified to the fashion store and payment completion information is received.

[0092] As described above, the system of the present invention allows users to receive personalized beauty and fashion suggestions using generative AI models, and easily make reservations and payments.

[0093] The processing flow will be explained below.

[0094] Step 1:

[0095] User: Launches the application and takes a photo of his / her face using the camera function or selects an existing photo. Clicks the button to upload a photo of his / her face.

[0096] Step 2:

[0097] Terminal: The facial photo data selected or taken by the user is temporarily stored within the application. An HTTP request is generated to send the stored facial photo data to the server. The generated HTTP request is sent to the server.

[0098] Step 3:

[0099] Server: Receives facial photo data sent from the device. The facial photo data is input into a generative AI model to analyze the user's facial features, contours, skin tone, etc. The analysis results are temporarily stored and used for subsequent processing.

[0100] Step 4:

[0101] User: Fills out a form to select their makeup, hairstyle and fashion preferences; answers choices and questions about current trends; completes the form and clicks the "Submit" button.

[0102] Step 5:

[0103] Terminal: The application temporarily stores the preferences and trend information entered by the user. It generates an HTTP request to send the stored data to the server. It then sends the generated HTTP request to the server.

[0104] Step 6:

[0105] Server: Receives preferences and trend information sent by the user. Combining the facial photo analysis results with preference information, the generative AI model generates optimal makeup, hairstyle, and fashion suggestions. The generated suggestions are temporarily stored.

[0106] Step 7:

[0107] Server: Sends the proposed results from the generative AI model to the terminal.

[0108] Step 8:

[0109] Terminal: Receives the proposal results sent from the server. The received proposal results are displayed within the application and confirmed by the user.

[0110] Step 9:

[0111] User: Check the suggestions and select a hair salon, beauty salon, or fashion store that suits them best. Enter the reservation information (date, time, service details, etc.) for the selected store and click the "Confirm reservation" button.

[0112] Step 10:

[0113] Terminal: The reservation information entered by the user is temporarily saved within the application. An HTTP request is generated to send the saved reservation information to the server. The generated HTTP request is sent to the server.

[0114] Step 11:

[0115] Server: Receives reservation information sent from the terminal. Generates and sends another HTTP request to the store's system to notify the corresponding store of the received reservation information. Receives reservation confirmation or adjustment information from the store and stores it.

[0116] Step 12:

[0117] Server: Sends reservation confirmation information to the terminal.

[0118] Step 13:

[0119] Terminal: Receives the reservation confirmation information sent from the server and displays it to the user. Generates a link to the payment platform and presents it to the user.

[0120] Step 14:

[0121] User: Click on the payment platform link displayed on the terminal, select the payment method, enter the required payment information, and click the "Complete Payment" button.

[0122] Step 15:

[0123] Terminal: Sends the payment information entered by the user to the payment platform. Receives the payment completion notification and generates an HTTP request to send to the server. Sends the generated HTTP request to the server.

[0124] Step 16:

[0125] Server: Receives the payment completion notification sent from the terminal. Based on the received information, sends a reservation completion and payment completion notification to the relevant store. Sends a completion notification to the terminal.

[0126] Step 17:

[0127] Terminal: Receives the reservation and payment completion notification sent from the server and displays it to the user.

[0128] Example 1

[0129] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0130] The present invention relates to a system that allows users to receive personalized beauty and fashion recommendations. In particular, the system analyzes a user's facial photograph and provides personalized recommendations, but the process is complicated, and making reservations and payments is often not easy. The challenge for such a system is to provide a user-friendly and efficient method for making reservations and payments.

[0131] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0132] In this invention, the server includes means for a user to upload a facial photo, means for transmitting the facial photo to the server, means for analyzing the facial photo using a generative AI model, means for generating suggestions tailored to the user's preferences based on the analysis results obtained by the generative AI model, means for transmitting the suggestions to a terminal, means for the user to make a reservation based on the suggestions, and means for making payment through a payment platform. This enables the user to receive personalized beauty and fashion suggestions using the generative AI model and to smoothly make reservations and payments.

[0133] "User" refers to an individual who uses the system to receive beauty and fashion suggestions.

[0134] "Means for uploading a facial photo" refers to a part of the system that provides a function for a user to take a new facial photo or select an existing facial photo using a smartphone or tablet and send it to the server.

[0135] "Means for sending facial photos to a server" refers to the function of transferring facial photo data uploaded from a terminal to a server using an HTTP request, etc.

[0136] A "generative AI model" refers to an algorithm or system that uses technologies such as deep learning to analyze facial photos and convert their features into numerical data.

[0137] "Means of generating suggestions that match the user's preferences based on the analysis results" refers to a function that combines facial photo feature data obtained using a generative AI model with preferences and trend information entered by the user to create optimal makeup, hairstyle, and fashion suggestions.

[0138] "Means for sending proposal results to a terminal" refers to a function for sending proposal content generated on a server to a user's terminal using a data format such as JSON.

[0139] The "means for making reservations" is a part of the system that includes a function that allows users to make online reservations for services at beauty salons, esthetic salons, fashion stores, etc. based on the suggestions.

[0140] "Means for making payments through a payment platform" means a part of the system that provides users with the ability to complete online payments for the services they select.

[0141] "Server" refers to a computer system that processes data received from a user's device and uses a generative AI model to analyze and make recommendations.

[0142] "Terminal" refers to a device used by a user to access the system, including smartphones and tablets.

[0143] An "HTTP request" refers to a communication protocol for sending data to a web server, and is used when sending data such as a photo of a person's face to a server.

[0144] "JSON format" is an abbreviation for JavaScript Object Notation, and refers to a format for structuring and describing data, and is used for sending proposal results, etc.

[0145] The present invention is a system that allows users to receive personalized beauty and fashion recommendations and smoothly make reservations and payments. Specific methods for implementing this system are described in detail below.

[0146] First, the user uses a device such as a smartphone or tablet. The user launches a dedicated application installed on the device and uploads a photo of their face through the application. Specifically, the user can take a new photo of their face using the camera function or select an existing photo from the gallery. The uploaded photo is temporarily stored on the device.

[0147] The device then sends the temporarily stored facial photo data to the server as an HTTP request. The server preprocesses the received facial photo data using an image processing algorithm and inputs this preprocessed data into the generative AI model. The generative AI model uses deep learning to convert the facial photo's features into numerical data and extract information such as contours, skin tone, and eye shape. The analysis results are stored in the server's internal database.

[0148] The server then retrieves the analysis results of the user's facial photo from its internal database and combines them with the preferences and current trends the user has entered through the application. Using this information, the generative AI model generates optimal makeup, hairstyle, and fashion recommendations. The generated recommendations are converted into JSON format and sent from the server to the device.

[0149] The user checks these suggestions on the device, selects their favorite hair salon, beauty salon, or fashion store, and makes a reservation. Specifically, the user enters the necessary information, such as the service content and desired date and time, into the reservation form within the application to confirm the reservation. The device sends the reservation information to the server, which then notifies the specified store of that information.

[0150] Finally, the user uses the payment platform to make an online payment. The terminal displays an interface for inputting payment information, and after the user inputs the required payment information, the payment is completed through the payment platform. The server receives a payment completion notification from the payment platform and sends the information to the terminal together with the reservation information.

[0151] For example, if a user is interested in natural makeup and a bob hairstyle, they can use the application to upload a photo of their face and input this information along with their desired style. The server uses a generative AI model to generate optimal makeup and hairstyle suggestions and presents them to the user. The user can then select a suggested salon, make a reservation, and complete payment online.

[0152] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0153] Program processing flow

[0154] Step 1:

[0155] Users launch the dedicated application on their smartphone, tablet, or other device, and can either take a new photo of their face using the camera function on the application's home screen, or select an existing photo from their gallery.

[0156] Input: User interaction via smartphone or tablet

[0157] Output: A face photo data to be uploaded

[0158] Step 2:

[0159] Users upload their facial photo data through the application interface, and by clicking the upload button, the facial photo data is temporarily saved on the device.

[0160] Input: Face photo data selected or taken by the user

[0161] Output: Temporarily saved face photo data

[0162] Step 3:

[0163] The device sends the temporarily stored facial photo data to the server in the form of an HTTP request. Specifically, it uses a REST API to send the data to the server in the form of multipart form data.

[0164] Input: Temporarily saved face photo data

[0165] Output: Face photo data sent to the server

[0166] Step 4:

[0167] The server inputs the received facial photo data into the generative AI model, checks the data format, and performs preprocessing if necessary before passing it to the AI ​​model.

[0168] Input: Face photo data received from the device

[0169] Output: Preprocessed facial photo data that is fed into the AI ​​model

[0170] Step 5:

[0171] The generative AI model analyzes facial photo data and extracts numerical data as features, such as facial contours, skin tone, and eye shape.

[0172] Input: Preprocessed facial photo data

[0173] Output: Extracted feature data

[0174] Step 6:

[0175] The server stores the feature data obtained from the generative AI model in an internal database, along with the beauty and fashion preferences and trend information entered by the user.

[0176] Input: Extracted feature data, user preferences and trend information

[0177] Output: Feature data and preference information stored in the internal database

[0178] Step 7:

[0179] The server retrieves feature data and preference information from an internal database, and then uses a generative AI model based on this to generate optimal makeup, hairstyle, and fashion suggestions.

[0180] Input: Feature data, user preferences and trend information

[0181] Output: Generated proposal data

[0182] Step 8:

[0183] The server converts the generated proposal data into JSON format and sends it to the device using a REST API, sending the encoded data packet to the device.

[0184] Input: Generated proposal data

[0185] Output: Proposal data in JSON format, data sent to the device

[0186] Step 9:

[0187] The device displays the received recommendation data to the user, who then checks the displayed recommendations and selects their preferred beauty salon, aesthetic salon, or fashion store.

[0188] Input: JSON formatted proposal data received from the server

[0189] Output: The suggestions displayed to the user

[0190] Step 10:

[0191] The user follows the suggestions, enters the necessary information (service details, date and time, etc.) into the reservation form within the application, and presses the reservation button to confirm the reservation.

[0192] Input: Booking information entered by the user

[0193] Output: Confirmed reservation information

[0194] Step 11:

[0195] The terminal sends the confirmed reservation information to the server, and the server notifies the store of the received reservation information.

[0196] Input: Reservation information confirmed by the user

[0197] Output: Reservation information sent to the server, reservation information notified to the target store

[0198] Step 12:

[0199] To make a payment using the payment platform, the user inputs the necessary payment information (such as card information) into the payment interface on the terminal, which then transmits this information to the payment platform for payment processing.

[0200] Input: User's payment information

[0201] Output: Payment information sent to the payment platform, payment processing results

[0202] Step 13:

[0203] The server receives a payment completion notification from the payment platform and transmits the information to the terminal together with the user's reservation information.

[0204] Input: Payment completion notification from payment platform

[0205] Output: Payment completion notification and reservation information sent to the user's terminal

[0206] Through these processing steps, users can receive personalized beauty and fashion recommendations using generative AI models, and make reservations and payments smoothly.

[0207] (Application example 1)

[0208] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0209] Systems that offer beauty and fashion recommendations are expected to enable users to quickly and accurately receive personalized recommendations, while also enabling smooth in-store reservations and payment. While conventional systems analyze users' facial photos to provide recommendations, these recommendations often do not adequately address individual preferences or trends. Furthermore, the reservation and payment processes are complicated, resulting in poor user convenience. Therefore, there is a need for a system that allows users to receive personalized recommendations and easily make reservations and payments.

[0210] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0211] In this invention, the server includes means for a user to upload a facial photo, means for transmitting the facial photo to the server, means for the server to analyze the facial photo using a generative AI model, means for suggesting makeup, hairstyles, and fashions that suit the user's preferences based on the analysis results obtained by the generative AI model, means for making a store reservation based on the suggested makeup, hairstyles, and fashions, means for making a payment through a payment platform, means for saving the user's data in an internal database, and interface means for checking the suggestion results on a smartphone. This allows the user to receive personalized beauty and fashion suggestions and smoothly make a reservation and make a payment at the suggested store.

[0212] "Means for users to upload facial photos" refers to the interface and functionality that allows users to select or take a photo of their face using a smartphone or other device and upload it into the application.

[0213] The "means for transmitting facial photographs to a server" refers to a communication protocol and interface for securely and efficiently transmitting uploaded facial photographs of users to a server.

[0214] "Means for the server to analyze facial photos using a generative AI model" refers to the function of inputting facial photo data received by the server into a generative AI model, which then analyzes facial features and extracts them as data.

[0215] "A means of suggesting makeup, hairstyles, and fashion that suit the user's preferences based on the analysis results obtained by a generative AI model" is a system that uses the results of facial photo analysis by a generative AI model to suggest optimal beauty and fashion options in combination with the user's preference information.

[0216] "Means for making store reservations based on suggested makeup, hairstyles, and fashion" refers to the interface and functionality for making reservations at hair salons, beauty salons, and fashion stores through the application based on the suggested results of the generative AI model.

[0217] "Means for making payments through a payment platform" means a payment system and interface for completing online payments for the booked services.

[0218] "Means for storing user data in an internal database" refers to a database system for securely storing user facial photographs and preference data for use in subsequent suggestions.

[0219] The "interface means for checking the proposal results on a smartphone" refers to a user interface that allows a user to visually check the beauty and fashion proposals from the generated AI model using a smartphone.

[0220] This invention is a system that allows users to receive personalized beauty and fashion suggestions, and smoothly make store reservations and payments. This system includes the following means.

[0221] A way for users to upload photos of their faces

[0222] The user starts the smartphone application, takes a photo of their face using the camera function or selects an existing photo from the gallery, and then uploads the photo data to the application through an interface for uploading a photo of their face.

[0223] A means of sending a face photo to the server

[0224] The device temporarily stores the uploaded facial photo data and prepares it for transmission to the server. The stored facial photo data is then sent to the server as an HTTP request. This is mainly done using the React Native HTTP request module.

[0225] The server generates a facial photo and analyzes it using an AI model.

[0226] The server inputs the facial photo data received from the device into the generative AI model. At this time, the server uses TensorFlow and PyTorch to analyze the facial photo's features (e.g., contours, skin tone, eye shape, etc.) and converts them into numerical data. The analysis results are then stored in an internal database (MongoDB) and transferred for subsequent processing.

[0227] How generative AI models generate proposals

[0228] The server retrieves the results of the user's facial photo analysis from an internal database, inputs the user's preferences and current trend information into the AI ​​model, and the generative AI model generates optimal suggestions for makeup, hairstyles, and fashion, and sends the generated suggestions in JSON format to the device.

[0229] A way to make restaurant reservations based on the proposed content

[0230] The user checks the suggestions and selects the hair salon, aesthetic salon, or fashion store that they like. They then enter reservation information, such as the service content and date and time of the selected store, and confirm the reservation.

[0231] A means of making payments through a payment platform

[0232] The terminal sends the reservation information entered by the user to the server. It then displays an interface for entering payment information, and the user enters the payment information. The entered payment information is sent to a payment platform (e.g., Stripe API) and the processing result is received. The server notifies the store of the reservation information received from the terminal, receives a payment completion notification from the payment platform, and confirms the completion of the reservation. The completed reservation and payment information are sent to the terminal.

[0233] Specific example explanation

[0234] For example, a user launches the "Beauty360" app on their smartphone and uploads a photo of their face. They then enter "I'm interested in natural makeup and bob hairstyles" through the interface. The server analyzes the photo and uses this information to generate optimal recommendations using a generative AI model. The user then confirms the recommendations, makes a reservation at the suggested salon, and makes payment via Stripe.

[0235] Example prompt sentence:

[0236] {

[0237] "face_features": {

[0238] "face_shape": "round",

[0239] "skin_tone": "fair",

[0240] "eye_shape": "almond"

[0241] },

[0242] "user_preference": {

[0243] "style": "natural",

[0244] "interest": ["hair", "makeup"]

[0245] },

[0246] "current_trends": {

[0247] "makeup": ["minimalist", "soft colors"],

[0248] "hair": ["long bob", "wavy"]

[0249] }

[0250] }

[0251] By writing and implementing it in this way, users will be able to receive personalized beauty and fashion suggestions, and easily make reservations and payments at physical stores using a smartphone application.

[0252] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0253] Step 1:

[0254] A user starts the smartphone application and takes a face photo using the camera function or selects an existing face photo from the gallery. This inputs face photo data. Next, the face photo data is uploaded to the application through the face photo upload interface and temporarily saved on the device.

[0255] Step 2:

[0256] The device sends the temporarily saved facial photo data to the server as an HTTP request. The server receives the facial photo data and prepares to start processing it as input data.

[0257] Step 3:

[0258] After receiving the facial photo data, the server uses TensorFlow to input the facial photo into a generative AI model. The generative AI model analyzes the facial photo's features (contours, skin tone, eye shape, etc.) and converts these features into numerical data. The converted data is stored in an internal database (e.g., MongoDB) and transferred for subsequent processing.

[0259] Step 4:

[0260] The server retrieves the results of the user's facial photo analysis from its internal database and inputs them into the generative AI model along with the user's preferences and trend information. The generative AI model then generates makeup, hairstyle, and fashion suggestions and outputs them in JSON format. The output suggestions are then sent to the device.

[0261] Step 5:

[0262] The user checks the recommendations through their smartphone application. At this time, the user interface visually displays the beauty recommendations created by the generative AI model. The user then selects their favorite beauty salon, beauty salon, or fashion store.

[0263] Step 6:

[0264] The user inputs the service details and date and time of the selected store and sends the reservation information to the server. The server notifies the corresponding store of the received reservation information. The notified store confirms the reservation and sends the reservation confirmation information to the server.

[0265] Step 7:

[0266] The terminal sends the reservation information entered by the user to the server, and then displays an interface for entering payment information. The user enters the payment information, and the terminal sends the information to a payment platform (e.g., Stripe API) for processing. The processing result is sent to the server.

[0267] Step 8:

[0268] The server receives a payment completion notification from the payment platform and confirms the completion of the reservation. This completion information is sent to the terminal so that the user can check it. This completes the entire process.

[0269] By following the above steps, users can receive personalized beauty and fashion recommendations and smoothly make reservations and payments. Examples of prompts are as follows:

[0270] {

[0271] "face_features": {

[0272] "face_shape": "round",

[0273] "skin_tone": "fair",

[0274] "eye_shape": "almond"

[0275] },

[0276] "user_preference": {

[0277] "style": "natural",

[0278] "interest": ["hair", "makeup"]

[0279] },

[0280] "current_trends": {

[0281] "makeup": ["minimalist", "soft colors"],

[0282] "hair": ["long bob", "wavy"]

[0283] }

[0284] }

[0285] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0286] The present invention is a system that provides beauty and fashion recommendations based on a user's facial photo and preferences, and allows for smooth reservations and payments. It also has the ability to recognize the user's emotional state and make optimal recommendations based on that. The following elements are required to implement this system:

[0287] A way for users to upload photos of their faces

[0288] User:

[0289] 1. Launch the application and take a photo of your face using the camera function or select an existing photo from your gallery.

[0290] 2. Upload your face photo data to the application through the interface for uploading face photos.

[0291] A means of sending a face photo to the server

[0292] Device:

[0293] 1. The uploaded facial photo data is temporarily saved and prepared for sending to the server.

[0294] 2. The saved facial photo data is sent to the server as an HTTP request.

[0295] The server generates a facial photo and analyzes it using an AI model.

[0296] server:

[0297] 1. The facial photo data received from the device is input into the generative AI model.

[0298] 2. The generative AI model analyzes the facial features (e.g., facial contours, skin tone, eye shape, etc.) and converts them into numerical data.

[0299] 3. The analysis results are stored in an internal database for subsequent processing.

[0300] Using an emotion engine to recognize user emotions

[0301] server:

[0302] 1. Input facial photo data into the emotion engine to analyze the user's emotional state (e.g., happiness, sadness, surprise, etc.).

[0303] 2. The analysis results of the emotion engine are stored in an internal database.

[0304] How generative AI models generate proposals

[0305] server:

[0306] 1. Obtain the analysis results and emotional state of the user's face photo from the internal database.

[0307] 2. User-input preferences and current trend information are fed into the AI ​​model to generate optimal makeup, hairstyle, and fashion suggestions.

[0308] 3. It also dynamically adjusts its suggestions based on your emotional state.

[0309] 4. The generated proposal results are sent to the device.

[0310] A way to make a reservation based on the suggestions

[0311] User:

[0312] 1. Check the suggestions and select your favorite hair salon, beauty salon, or fashion store.

[0313] 2. Enter reservation information such as the service content, date and time of the selected store and confirm your reservation.

[0314] A means of making payments through a payment platform

[0315] Device:

[0316] 1. The reservation information entered by the user is sent to the server.

[0317] 2. An interface for entering payment information is displayed, and the user enters the payment information.

[0318] 3. The entered payment information is sent to the payment platform and the processing result is received.

[0319] server:

[0320] 1. Notify beauty salons, esthetic salons, and fashion stores of reservation information received from the device.

[0321] 2. Receive a payment confirmation from the payment platform to confirm the completion of your booking.

[0322] 3. Send the completed booking and payment information to the terminal.

[0323] Specific examples

[0324] Example 1: Makeup and hairstyle suggestions

[0325] User:

[0326] 1. Upload a photo of your face using the application and indicate that you are interested in natural makeup and bob hairstyles.

[0327] 2. Check the suggestions from the server and make a reservation at the suggested beauty salon.

[0328] Device:

[0329] 1. Send a face photo and user preferences to the server.

[0330] 2. Receive suggestions from the server and present them to the user.

[0331] 3. Send the reservation information to the server and display the payment interface.

[0332] server:

[0333] 1. Analyze your facial photo and suggest the best makeup and hairstyle.

[0334] 2. The emotion engine analyzes the user's emotional state and adjusts the suggestions accordingly.

[0335] 3. Notify the beauty salon of the user's reservation information and receive payment completion information.

[0336] Example 2: Fashion coordination suggestions

[0337] User:

[0338] 1. Enter your interest in casual style and upload a photo of your face.

[0339] 2. Make an appointment at the suggested fashion store and complete the payment online.

[0340] Device:

[0341] 1. Send your face photo and preference information to the server.

[0342] 2. Receive suggestions from the server and present them to the user.

[0343] 3. Send the reservation information to the server and process the payment.

[0344] server:

[0345] 1. We suggest the best fashion coordination based on the analysis of your face photo and your preferences.

[0346] 2. The emotion engine analyzes the user's emotional state and adjusts the suggestions accordingly.

[0347] 3. The user's reservation information is notified to the fashion store and payment completion information is received.

[0348] As described above, the system of the present invention allows users to receive personalized beauty and fashion suggestions using generative AI models and emotion engines, and to easily make reservations and payments.

[0349] The processing flow will be explained below.

[0350] Step 1:

[0351] User: Launches the application and takes a photo of his / her face using the camera function or selects an existing photo from the gallery. Clicks the button to upload a photo of his / her face.

[0352] Step 2:

[0353] Terminal: The facial photo data selected or taken by the user is temporarily stored within the application. An HTTP request is generated to send the stored facial photo data to the server. The generated HTTP request is sent to the server.

[0354] Step 3:

[0355] Server: Receives facial photo data sent from the device. Preprocesses the facial photo data to pass it to the generative AI model and emotion engine.

[0356] Step 4:

[0357] Server: The generative AI model analyzes the facial features (e.g., facial contours, skin tone, eye shape, etc.) and converts them into numerical data. The analysis results are stored in an internal database.

[0358] Step 5:

[0359] Server: Inputs a facial photo into the emotion engine and analyzes the user's emotional state (e.g., happiness, sadness, surprise, etc.). The analysis results of the emotion engine are also stored in an internal database.

[0360] Step 6:

[0361] User: Fills out a form to select their makeup, hairstyle and fashion preferences; answers choices and questions about current trends; completes the form and clicks the "Submit" button.

[0362] Step 7:

[0363] Terminal: The application temporarily stores the preferences and trend information entered by the user. It generates an HTTP request to send the stored data to the server. It then sends the generated HTTP request to the server.

[0364] Step 8:

[0365] Server: Receives preferences and trend information sent by the user. Combining the results of facial photo analysis, emotional state, and preference information, the generative AI model generates optimal makeup, hairstyle, and fashion suggestions. The suggestions are dynamically adjusted based on the user's emotional state.

[0366] Step 9:

[0367] Server: Sends the proposed results from the generative AI model to the terminal.

[0368] Step 10:

[0369] Terminal: Receives the proposal results sent from the server. The received proposal results are displayed within the application and confirmed by the user.

[0370] Step 11:

[0371] User: Check the suggestions and select a hair salon, beauty salon, or fashion store that suits them best. Enter the reservation information (date, time, service details, etc.) for the selected store and click the "Confirm reservation" button.

[0372] Step 12:

[0373] Terminal: The reservation information entered by the user is temporarily saved within the application. An HTTP request is generated to send the reservation information to the server. The generated HTTP request is sent to the server.

[0374] Step 13:

[0375] Server: Receives reservation information sent from the terminal. Generates and sends another HTTP request to the store's system to notify the corresponding store of the received reservation information. Receives reservation confirmation or adjustment information from the store and stores it.

[0376] Step 14:

[0377] Server: Sends reservation confirmation information to the terminal.

[0378] Step 15:

[0379] Terminal: Receives the reservation confirmation information sent from the server and displays it to the user. Generates a link to the payment platform and presents it to the user.

[0380] Step 16:

[0381] User: Click on the payment platform link displayed on the terminal, select the payment method, enter the required payment information, and click the "Complete Payment" button.

[0382] Step 17:

[0383] Terminal: Sends the payment information entered by the user to the payment platform. Receives the payment completion notification and generates an HTTP request to send to the server. Sends the generated HTTP request to the server.

[0384] Step 18:

[0385] Server: Receives the payment completion notification sent from the terminal. Based on the received information, sends a reservation completion and payment completion notification to the relevant store. Sends a completion notification to the terminal.

[0386] Step 19:

[0387] Terminal: Receives the reservation and payment completion notification sent from the server and displays it to the user.

[0388] Example 2

[0389] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0390] Conventional beauty and fashion recommendation systems have difficulty providing personalized recommendations that comprehensively consider a user's facial photo, emotional state, preferences, etc. Furthermore, there has been a lack of systems that allow users to instantly complete reservations and payments based on the recommendations. Therefore, there has been a strong demand for the development of a system that allows users to easily receive optimal beauty and fashion recommendations and quickly use the service.

[0391] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0392] In this invention, the server includes means for using an emotion engine that analyzes the user's emotional state, means for adjusting the content of suggestions according to the emotional state, and means for suggesting makeup, hairstyles, and fashions that suit the user's preferences based on the analysis results obtained by the generative AI model. This makes it possible to quickly provide personalized and optimal suggestions by comprehensively considering the user's facial photo and emotional state, and to smoothly execute the reservation and payment processes.

[0393] A "user" is a person who uses the system to upload a photo of their face and receive beauty and fashion suggestions.

[0394] A "face photo" is still image data that includes features such as the user's facial contours, eye shape, and skin tone.

[0395] The "server" is a central processing unit that processes facial photo data, user preferences, and emotional state, and generates optimal suggestions using generative AI models and emotion engines.

[0396] A "generative AI model" is an artificial intelligence model that analyzes facial photos and converts their features into numerical data.

[0397] An "emotion engine" is software that analyzes a user's emotional state (e.g., happiness, sadness, surprise, etc.) from a photograph of their face.

[0398] "Makeup" refers to methods and products for applying makeup to a user's face.

[0399] A "hairstyle" is a method or design for instructing the shape and arrangement of a user's hair.

[0400] "Fashion" refers to a user's clothing, accessories, and other decorative items related to their appearance.

[0401] A "reservation" is a procedure in which a user determines in advance the date, time, and content of services to be used at a beauty salon, esthetic salon, or fashion store.

[0402] "Payment Platform" means an online payment system through which Users pay for Services.

[0403] "Suggestions" are specific advice and options regarding makeup, hairstyles, and fashion provided to users based on the results of analysis by generative AI models and emotion engines.

[0404] MODE FOR CARRYING OUT THE INVENTION

[0405] This system provides beauty and fashion recommendations based on a user's facial photograph and preferences, and allows for smooth booking and payment. It also has the ability to recognize the user's emotional state and provide optimal recommendations based on that. The specific configuration and operating procedures for implementing this system are described below.

[0406] A way for users to upload photos of their faces

[0407] The user launches the application on their smartphone or PC and takes a photo of their face using the camera function, or selects an existing photo from the gallery. Then, the user presses the upload button on the application to upload the photo data to the application. This application can run on the iOS or Android platform, for example.

[0408] A means of sending a face photo to the server

[0409] The device temporarily stores the uploaded facial photo data and sends it to the server in the form of an HTTP request. The HTTP request header contains authentication information, and the body contains the facial photo data. The device's local storage is used for temporary storage.

[0410] The server generates a facial photo and analyzes it using an AI model.

[0411] The server inputs the facial photo data received from the device into a generative AI model. The server performs preprocessing on the facial photo data, such as resizing and color normalization. The data is then input into a specialized facial analysis AI model (e.g., a convolutional neural network using TensorFlow) to convert features such as facial contours, skin tone, and eye shape into numerical data. The results of this analysis are stored in an internal database and used for subsequent processing.

[0412] Using an emotion engine to recognize user emotions

[0413] The server inputs the facial photo data into an emotion engine to analyze the user's emotional state (e.g., happiness, sadness, surprise, etc.). The emotion engine's analysis results are also stored in an internal database. The emotion engine uses, for example, facial expression recognition software (e.g., Microsoft's Azure Emotion API).

[0414] How generative AI models generate proposals

[0415] The server retrieves the analysis results of the user's facial photo and emotional state from an internal database, inputs the user's preferences and current trend information into the AI ​​model, and generates optimal suggestions for makeup, hairstyles, and fashion. Furthermore, the suggestions are dynamically adjusted according to the user's emotional state. The results of these suggestions are returned to the device in JSON format.

[0416] For example, if a user enters "I'm interested in natural makeup and bob hairstyles" and uploads a photo of their face, the server will use a generative AI model to analyze the user's facial features and make optimal suggestions for natural makeup and bob hairstyles based on those features.

[0417] Example prompt sentence:

[0418] "I'm interested in natural makeup and bob hairstyles."

[0419] A way to make a reservation based on the suggestions

[0420] The user reviews the suggestions and selects their favorite beauty salon, esthetic salon, or fashion store on the application. They then enter reservation information such as the service content and date and time of the selected store and press "Confirm reservation" to make the reservation. The reservation information is then sent from the device to the server.

[0421] A means of making payments through a payment platform

[0422] The terminal sends the reservation information entered by the user to the server, and also displays an interface for entering payment information, where the user enters payment information such as credit card information. The terminal sends data to the API of the payment platform (e.g., Stripe or PayPal) and receives the processing results.

[0423] A means by which the server processes booking and payment information

[0424] The server notifies the beauty salon, esthetic salon, or fashion store of the reservation information received from the device. It also receives a payment completion notification from the payment platform and confirms the completion of the reservation. The completed reservation and payment information are returned to the device in JSON format.

[0425] As described above, the system of the present invention allows users to receive personalized beauty and fashion suggestions using generative AI models and emotion engines, and to easily make reservations and payments.

[0426] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0427] Step 1:

[0428] Users launch the application on their smartphone or PC, take a photo of their face using the camera function, or select an existing photo from the gallery, and then press the upload button to upload the photo data to the application.

[0429] Input: Facial photo data taken or selected by the user

[0430] Output: Facial photo data sent to the application through the upload interface

[0431] Specific actions: The user taps the app's camera icon to launch the camera, takes a photo of their face, and taps "Done." Alternatively, the user taps the gallery icon, selects an existing photo, and presses the "Upload" button.

[0432] Step 2:

[0433] The device temporarily stores the uploaded facial photo data and sends it to the server in the form of an HTTP request.

[0434] Input: Uploaded face photo data

[0435] Output: Facial photo data sent to the server in the form of an HTTP request

[0436] Specific operation: The device temporarily stores the face photo data in local storage and then sends it to the server's API endpoint in the background. The HTTP request header contains authentication information, and the body contains the face photo data.

[0437] Step 3:

[0438] The server inputs the facial photo data received from the device into the generative AI model. The server preprocesses the facial photo data by resizing and color normalizing it, and then inputs the data into the generative AI model.

[0439] Input: Facial photo data received from the device

[0440] Output: Analysis results converted from facial features into numerical data

[0441] Specific operation: The server performs preprocessing on the facial photo data, such as resizing and color normalizing, and then inputs the data into a generative AI model (e.g., a convolutional neural network using TensorFlow), converting features such as facial contours, skin tone, and eye shape into numerical data.

[0442] Step 4:

[0443] The server inputs the facial photo data into an emotion engine to analyze the user's emotional state (e.g., happiness, sadness, surprise, etc.).

[0444] Input: Facial photo data

[0445] Output: Analysis results showing the user's emotional state

[0446] Specific operation: The server inputs facial photo data into an emotion engine (e.g., facial expression recognition software), analyzes the emotional state, and stores the results in an internal database.

[0447] Step 5:

[0448] The server retrieves the analysis results of the user's facial photo and emotional state from an internal database, and inputs the user's input preferences and current trend information into an AI model to generate optimal suggestions for makeup, hairstyles, and fashion.

[0449] Input: User's facial photo analysis results, emotional state, preferences and trend information

[0450] Output: Makeup, hairstyle, and fashion suggestions

[0451] Specific operation: The server retrieves the necessary data from the internal database, integrates them, and inputs them into the generative AI model. The generative AI model generates a proposal, adjusts the proposal content using the emotion engine, and sends it to the device in JSON format.

[0452] Step 6:

[0453] The user checks the suggestions on the application and selects the hair salon, beauty salon, or fashion store that suits them best. After that, they enter reservation information such as the service content, date and time, and confirm the reservation.

[0454] Input: Proposal details, reservation information (service details, date and time)

[0455] Output: Confirmed reservation information

[0456] Specific operation: The user checks the list of suggestions, taps the selection button to move to the reservation page, enters the service details and date and time, and taps "Confirm reservation."

[0457] Step 7:

[0458] The terminal transmits the reservation information entered by the user to the server, displays an interface for entering payment information, and the user enters the payment information. The terminal then transmits the data to the payment platform and receives the processing result.

[0459] Input: Reservation information, payment information

[0460] Output: Processing result by payment platform

[0461] Specific operation: The terminal sends the reservation information to the server, then displays the payment page. The user enters credit card information and taps the "Pay" button. The terminal then sends the data to the payment platform's API and receives the processing result.

[0462] Step 8:

[0463] The server notifies the beauty salon, esthetic salon, or fashion store of the reservation information received from the terminal, receives a payment completion notification from the payment platform, confirms the completion of the reservation, and sends the completed reservation and payment information to the terminal.

[0464] Input: Reservation information, payment completion notification

[0465] Output: Reservation completion information, payment completion information

[0466] Specific operation: The server notifies the store management system of the reservation information in real time, then receives a payment completion notification from the payment platform, updates the reservation status to "Completed", and returns the reservation and payment completion information to the terminal in JSON format.

[0467] (Application example 2)

[0468] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0469] Conventional beauty and fashion recommendation systems were unable to provide personalized recommendations that fully considered the individual characteristics and emotional state of each user. As a result, users were often dissatisfied with the recommendations and stopped using the service. In addition, the reservation and payment procedures were cumbersome, leaving a need for an improved user experience.

[0470] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0471] In this invention, the server includes means for a user to upload a facial photo, means for transmitting the facial photo to the server, means for the server to use a generative AI model that analyzes the facial photo, means for suggesting beauty and fashion that matches the user's preferences based on the analysis results obtained by the generative AI model, means for making a reservation at a service facility based on the suggested beauty and fashion, means for making a payment through a payment platform, means for inputting the user's facial photo and preference information through a smartphone application, means for recognizing the user's emotional state using an emotion engine, and means for dynamically adjusting the content of the proposals according to the emotional state. This not only enables personalized proposals based on the user's individual characteristics and emotional state, but also enables a smooth process from reservation to payment.

[0472] "Means for users to upload facial photos" refers to the methods or functions that allow users to take or select their own facial photos and transmit the photo data via the Internet.

[0473] "Means for sending facial photos to a server" refers to a system that has the function of temporarily storing facial photos taken or selected by a user and sending them to an external server via the Internet.

[0474] "Means of using generative AI models" refers to methods or functions that use models to generate results by using artificial intelligence to analyze collected data.

[0475] "Means of suggesting beauty and fashion that suits the user's preferences" refers to a function that suggests beauty, hairstyles, and fashion that suits the user's individual preferences based on the analysis results obtained by the generative AI model.

[0476] "Means for making reservations at service providing facilities" refers to a function that enables users to make online reservations at service providing facilities such as beauty salons, esthetic salons, and fashion stores based on the beauty and fashion recommendations.

[0477] "Means of making payments through a payment platform" refers to the use of payment systems and services to make payments for online reservations in a secure and efficient manner.

[0478] "Means for inputting a user's facial photograph and preferred information via a smartphone application" refers to an interface that allows a user to input a photograph of their face and preferred beauty and fashion information via an application that runs on a smartphone.

[0479] "Means for recognizing a user's emotional state using an emotion engine" refers to a method that uses a system or algorithm to analyze a facial photograph and determine the user's current emotional state (such as joy, sadness, surprise, etc.).

[0480] "Means for dynamically adjusting the content of suggestions according to the emotional state" refers to a function that appropriately changes and adjusts the content of beauty and fashion suggestions in real time based on the recognized emotional state of the user.

[0481] The present invention is a system in which a user uploads a photo of their face via a smartphone application, and the server analyzes the photo using a generated AI model to suggest beauty and fashion products that match the user's preferences. This system is realized through the following configuration and processing.

[0482] The server first receives a facial photo taken or selected by the user using a smartphone application. The facial photo is uploaded using the smartphone's camera or gallery function. This photo data is then sent to the server via the Internet.

[0483] The server temporarily stores the received facial photo data and analyzes it using a generative AI model. The analysis extracts facial features such as facial contours, skin tone, and eye shape as numerical data. This process uses machine learning frameworks such as TensorFlow.

[0484] Furthermore, the server uses an emotion engine to recognize the user's emotional state. The emotion engine can determine the user's current emotion (e.g., joy, sadness, surprise, etc.) from the facial photograph. This information is also stored in an internal database as numerical data.

[0485] The server uses a generative AI model to generate optimal beauty and fashion recommendations based on the user's preferences, current trends, and the acquired facial features and emotional state. The recommendations are dynamically adjusted according to the user's emotional state and presented in a form optimized for each individual user. The recommendations are displayed to the user via a smartphone application.

[0486] Users can check the suggestions and make reservations at beauty salons, esthetic salons, and fashion stores that they are interested in. They enter reservation information through the smartphone application and confirm the reservation. The reservation information is sent to the server, which then notifies the relevant service provider.

[0487] Finally, payment is processed. The smartphone application connects to a payment platform and securely processes the payment information entered by the user. Services such as Stripe can be used as payment platforms. A notification of payment completion is sent to the server, confirming the completion of the reservation.

[0488] As a concrete example, consider the following prompt sentence:

[0489] Example prompt sentence:

[0490] "User's face photo has been uploaded and preprocessed. Generate a personalized beauty or fashion suggestion based on the following parameters:

[0491] Face features: [numerical data such as face shape, skin tone, eye shape, etc.]

[0492] Emotion state: "Joy"

[0493] User preferences: ["Natural makeup", "Bob hairstyle"]

[0494] Current trends: ["Spring 2023 trends"]

[0495] This will enable users to receive beauty and fashion suggestions based on their facial photos, preferences, and real-time emotional state, and will also enable them to complete reservations and payments on the spot.

[0496] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0497] Step 1:

[0498] The user starts the smartphone application and takes a photo of their face or selects one from the gallery. They then use the application's camera or gallery function to obtain a photo of their face and press the upload button to enter the photo data. The entered photo data is temporarily saved.

[0499] Step 2:

[0500] The device sends face photo data to the server. The face photo data is temporarily saved and then sent to the server via an HTTP request. The input in this step is the face photo data, and the output is the data sent to the server.

[0501] Step 3:

[0502] The server inputs the received facial photo data into a generative AI model for analysis. Using a machine learning framework such as TensorFlow, facial photo features (contours, skin tone, eye shape, etc.) are extracted as numerical data. The input is facial photo data, and the output is numerical data indicating facial features.

[0503] Step 4:

[0504] The server uses an emotion engine to recognize the user's emotional state. The received facial photo data is input into the emotion engine to determine the user's emotional state (e.g., joy, sadness, surprise, etc.). The emotion engine's specific operation uses an algorithm to analyze facial expressions and micro-expressions. The input is facial photo data, and the output is data indicating the user's emotional state.

[0505] Step 5:

[0506] The server generates beauty and fashion suggestions using a generative AI model based on facial feature data, emotional state data, user preference information, and trend information. This uses prompts that take into account the user's input preferences (e.g., natural makeup, bob hairstyle) and current trend information. Specific examples of generated prompts include:

[0507] "User's face photo has been uploaded and preprocessed. Generate a personalized beauty or fashion suggestion based on the following parameters:

[0508] Face features: [numerical data such as face shape, skin tone, eye shape, etc.]

[0509] Emotion state: "Joy"

[0510] User preferences: ["Natural makeup", "Bob hairstyle"]

[0511] Current trends: ["Spring 2023 trends"]

[0512] The inputs are facial feature data, emotional state data, user preference information, and trend information, and the output is suggested beauty and fashion content.

[0513] Step 6:

[0514] The user checks the proposed beauty and fashion details on a smartphone application and selects the service provider of interest (beauty salon, esthetic salon, fashion store). The user then enters the reservation information for the desired service (date, time, location, service details, etc.) and presses the reservation button. The input is the confirmation result of the proposal details and the reservation information, and the output is the entered reservation information.

[0515] Step 7:

[0516] The terminal sends the entered reservation information to the server, which then notifies the relevant service provider of that information. The input is the reservation information, and the output is a notification to the service provider.

[0517] Step 8:

[0518] A user enters payment information into a smartphone application to make a payment. The application then sends the entered payment information to a payment platform (e.g., Stripe) to process the payment. The input is the payment information, and the output is the payment processing result.

[0519] Step 9:

[0520] The server receives the payment completion notification and confirms the completion of the reservation. It then notifies the user of the completed reservation information and payment information. The input is the payment processing result, and the output is the completed reservation information and payment completion notification.

[0521] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0522] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0523] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0524] [Second embodiment]

[0525] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0526] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0527] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0528] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0529] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0530] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0531] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0532] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0533] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0534] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0535] In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0536] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0537] The present invention is a system that allows users to receive personalized beauty and fashion recommendations and smoothly make reservations and payments. The following elements are required to implement this system:

[0538] A way for users to upload photos of their faces

[0539] User:

[0540] 1. Launch the application and take a photo of your face using the camera function or select an existing photo from your gallery.

[0541] 2. Upload the facial photo data to the application through the interface for uploading facial photos.

[0542] A means of sending a face photo to the server

[0543] Device:

[0544] 1. The uploaded facial photo data is temporarily saved and prepared for sending to the server.

[0545] 2. The saved facial photo data is sent to the server as an HTTP request.

[0546] The server generates a facial photo and analyzes it using an AI model.

[0547] server:

[0548] 1. The facial photo data received from the device is input into the generative AI model.

[0549] 2. The generative AI model analyzes the facial features (e.g., facial contours, skin tone, eye shape, etc.) and converts them into numerical data.

[0550] 3. Store the analysis results in an internal database and forward them for further processing.

[0551] How generative AI models generate proposals

[0552] server:

[0553] 1. Obtain the analysis results of the user's facial photo from the internal database.

[0554] 2. User-input preferences and current trend information are fed into the AI ​​model to generate optimal makeup, hairstyle, and fashion suggestions.

[0555] 3. The generated proposal results are sent to the device in JSON format.

[0556] A way to make a reservation based on the suggestions

[0557] User:

[0558] 1. Check the suggestions and select your favorite hair salon, beauty salon, or fashion store.

[0559] 2. Enter reservation information such as the service content, date and time of the selected store and confirm your reservation.

[0560] A means of making payments through a payment platform

[0561] Device:

[0562] 1. The reservation information entered by the user is sent to the server.

[0563] 2. An interface for entering payment information is displayed, and the user enters the payment information.

[0564] 3. The entered payment information is sent to the payment platform and the processing result is received.

[0565] server:

[0566] 1. Notify beauty salons, esthetic salons, and fashion stores of reservation information received from the device.

[0567] 2. Receive a payment confirmation from the payment platform to confirm the completion of your booking.

[0568] 3. Send the completed booking and payment information to the terminal.

[0569] Specific examples

[0570] Example 1: Makeup and hairstyle suggestions

[0571] User:

[0572] 1. Upload a photo of your face using the application and indicate that you are interested in natural makeup and bob hairstyles.

[0573] 2. Check the suggestions from the server and make a reservation at the suggested beauty salon.

[0574] Device:

[0575] 1. Send a face photo and user preferences to the server.

[0576] 2. Receive suggestions from the server and present them to the user.

[0577] 3. Send the reservation information to the server and display the payment interface.

[0578] server:

[0579] 1. Analyze your facial photo and suggest the best makeup and hairstyle.

[0580] 2. Notify the beauty salon of the user's reservation information and receive payment completion information.

[0581] Example 2: Fashion coordination suggestions

[0582] User:

[0583] 1. Enter your interest in casual style and upload a photo of your face.

[0584] 2. Make an appointment at the suggested fashion store and complete the payment online.

[0585] Device:

[0586] 1. Send your face photo and preference information to the server.

[0587] 2. Receive suggestions from the server and present them to the user.

[0588] 3. Send the reservation information to the server and process the payment.

[0589] server:

[0590] 1. We suggest the best fashion coordination based on the analysis of your face photo and your preferences.

[0591] 2. The user's reservation information is notified to the fashion store and payment completion information is received.

[0592] As described above, the system of the present invention allows users to receive personalized beauty and fashion suggestions using generative AI models, and easily make reservations and payments.

[0593] The processing flow will be explained below.

[0594] Step 1:

[0595] User: Launches the application and takes a photo of his / her face using the camera function or selects an existing photo. Clicks the button to upload a photo of his / her face.

[0596] Step 2:

[0597] Terminal: The facial photo data selected or taken by the user is temporarily stored within the application. An HTTP request is generated to send the stored facial photo data to the server. The generated HTTP request is sent to the server.

[0598] Step 3:

[0599] Server: Receives facial photo data sent from the device. The facial photo data is input into a generative AI model to analyze the user's facial features, contours, skin tone, etc. The analysis results are temporarily stored and used for subsequent processing.

[0600] Step 4:

[0601] User: Fills out a form to select their makeup, hairstyle and fashion preferences; answers choices and questions about current trends; completes the form and clicks the "Submit" button.

[0602] Step 5:

[0603] Terminal: The application temporarily stores the preferences and trend information entered by the user. It generates an HTTP request to send the stored data to the server. It then sends the generated HTTP request to the server.

[0604] Step 6:

[0605] Server: Receives preferences and trend information sent by the user. Combining the facial photo analysis results with preference information, the generative AI model generates optimal makeup, hairstyle, and fashion suggestions. The generated suggestions are temporarily stored.

[0606] Step 7:

[0607] Server: Sends the proposed results from the generative AI model to the terminal.

[0608] Step 8:

[0609] Terminal: Receives the proposal results sent from the server. The received proposal results are displayed within the application and confirmed by the user.

[0610] Step 9:

[0611] User: Check the suggestions and select a hair salon, beauty salon, or fashion store that suits them best. Enter the reservation information (date, time, service details, etc.) for the selected store and click the "Confirm reservation" button.

[0612] Step 10:

[0613] Terminal: The reservation information entered by the user is temporarily saved within the application. An HTTP request is generated to send the saved reservation information to the server. The generated HTTP request is sent to the server.

[0614] Step 11:

[0615] Server: Receives reservation information sent from the terminal. Generates and sends another HTTP request to the store's system to notify the corresponding store of the received reservation information. Receives reservation confirmation or adjustment information from the store and stores it.

[0616] Step 12:

[0617] Server: Sends reservation confirmation information to the terminal.

[0618] Step 13:

[0619] Terminal: Receives the reservation confirmation information sent from the server and displays it to the user. Generates a link to the payment platform and presents it to the user.

[0620] Step 14:

[0621] User: Click on the payment platform link displayed on the terminal, select the payment method, enter the required payment information, and click the "Complete Payment" button.

[0622] Step 15:

[0623] Terminal: Sends the payment information entered by the user to the payment platform. Receives the payment completion notification and generates an HTTP request to send to the server. Sends the generated HTTP request to the server.

[0624] Step 16:

[0625] Server: Receives the payment completion notification sent from the terminal. Based on the received information, sends a reservation completion and payment completion notification to the relevant store. Sends a completion notification to the terminal.

[0626] Step 17:

[0627] Terminal: Receives the reservation and payment completion notification sent from the server and displays it to the user.

[0628] Example 1

[0629] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0630] The present invention relates to a system that allows users to receive personalized beauty and fashion recommendations. In particular, the system analyzes a user's facial photograph and provides personalized recommendations, but the process is complicated, and making reservations and payments is often not easy. The challenge for such a system is to provide a user-friendly and efficient method for making reservations and payments.

[0631] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0632] In this invention, the server includes means for a user to upload a facial photo, means for transmitting the facial photo to the server, means for analyzing the facial photo using a generative AI model, means for generating suggestions tailored to the user's preferences based on the analysis results obtained by the generative AI model, means for transmitting the suggestions to a terminal, means for the user to make a reservation based on the suggestions, and means for making payment through a payment platform. This enables the user to receive personalized beauty and fashion suggestions using the generative AI model and to smoothly make reservations and payments.

[0633] "User" refers to an individual who uses the system to receive beauty and fashion suggestions.

[0634] "Means for uploading a facial photo" refers to a part of the system that provides a function for a user to take a new facial photo or select an existing facial photo using a smartphone or tablet and send it to the server.

[0635] "Means for sending facial photos to a server" refers to the function of transferring facial photo data uploaded from a terminal to a server using an HTTP request, etc.

[0636] A "generative AI model" refers to an algorithm or system that uses technologies such as deep learning to analyze facial photos and convert their features into numerical data.

[0637] "Means of generating suggestions that match the user's preferences based on the analysis results" refers to a function that combines facial photo feature data obtained using a generative AI model with preferences and trend information entered by the user to create optimal makeup, hairstyle, and fashion suggestions.

[0638] "Means for sending proposal results to a terminal" refers to a function for sending proposal content generated on a server to a user's terminal using a data format such as JSON.

[0639] The "means for making reservations" is a part of the system that includes a function that allows users to make online reservations for services at beauty salons, esthetic salons, fashion stores, etc. based on the suggestions.

[0640] "Means for making payments through a payment platform" means a part of the system that provides users with the ability to complete online payments for the services they select.

[0641] "Server" refers to a computer system that processes data received from a user's device and uses a generative AI model to analyze and make recommendations.

[0642] "Terminal" refers to a device used by a user to access the system, including smartphones and tablets.

[0643] An "HTTP request" refers to a communication protocol for sending data to a web server, and is used when sending data such as a photo of a person's face to a server.

[0644] "JSON format" is an abbreviation for JavaScript Object Notation, and refers to a format for structuring and describing data, and is used for sending proposal results, etc.

[0645] The present invention is a system that allows users to receive personalized beauty and fashion recommendations and smoothly make reservations and payments. Specific methods for implementing this system are described in detail below.

[0646] First, the user uses a device such as a smartphone or tablet. The user launches a dedicated application installed on the device and uploads a photo of their face through the application. Specifically, the user can take a new photo of their face using the camera function or select an existing photo from the gallery. The uploaded photo is temporarily stored on the device.

[0647] The device then sends the temporarily stored facial photo data to the server as an HTTP request. The server preprocesses the received facial photo data using an image processing algorithm and inputs this preprocessed data into the generative AI model. The generative AI model uses deep learning to convert the facial photo's features into numerical data and extract information such as contours, skin tone, and eye shape. The analysis results are stored in the server's internal database.

[0648] The server then retrieves the analysis results of the user's facial photo from its internal database and combines them with the preferences and current trends the user has entered through the application. Using this information, the generative AI model generates optimal makeup, hairstyle, and fashion recommendations. The generated recommendations are converted into JSON format and sent from the server to the device.

[0649] The user checks these suggestions on the device, selects their favorite hair salon, beauty salon, or fashion store, and makes a reservation. Specifically, the user enters the necessary information, such as the service content and desired date and time, into the reservation form within the application to confirm the reservation. The device sends the reservation information to the server, which then notifies the specified store of that information.

[0650] Finally, the user uses the payment platform to make an online payment. The terminal displays an interface for inputting payment information, and after the user inputs the required payment information, the payment is completed through the payment platform. The server receives a payment completion notification from the payment platform and sends the information to the terminal together with the reservation information.

[0651] For example, if a user is interested in natural makeup and a bob hairstyle, they can use the application to upload a photo of their face and input this information along with their desired style. The server uses a generative AI model to generate optimal makeup and hairstyle suggestions and presents them to the user. The user can then select a suggested salon, make a reservation, and complete payment online.

[0652] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0653] Program processing flow

[0654] Step 1:

[0655] Users launch the dedicated application on their smartphone, tablet, or other device, and can either take a new photo of their face using the camera function on the application's home screen, or select an existing photo from their gallery.

[0656] Input: User interaction via smartphone or tablet

[0657] Output: A face photo data to be uploaded

[0658] Step 2:

[0659] Users upload their facial photo data through the application interface, and by clicking the upload button, the facial photo data is temporarily saved on the device.

[0660] Input: Face photo data selected or taken by the user

[0661] Output: Temporarily saved face photo data

[0662] Step 3:

[0663] The device sends the temporarily stored facial photo data to the server in the form of an HTTP request. Specifically, it uses a REST API to send the data to the server in the form of multipart form data.

[0664] Input: Temporarily saved face photo data

[0665] Output: Face photo data sent to the server

[0666] Step 4:

[0667] The server inputs the received facial photo data into the generative AI model, checks the data format, and performs preprocessing if necessary before passing it to the AI ​​model.

[0668] Input: Face photo data received from the device

[0669] Output: Preprocessed facial photo data that is fed into the AI ​​model

[0670] Step 5:

[0671] The generative AI model analyzes facial photo data and extracts numerical data as features, such as facial contours, skin tone, and eye shape.

[0672] Input: Preprocessed facial photo data

[0673] Output: Extracted feature data

[0674] Step 6:

[0675] The server stores the feature data obtained from the generative AI model in an internal database, along with the beauty and fashion preferences and trend information entered by the user.

[0676] Input: Extracted feature data, user preferences and trend information

[0677] Output: Feature data and preference information stored in the internal database

[0678] Step 7:

[0679] The server retrieves feature data and preference information from an internal database, and then uses a generative AI model based on this to generate optimal makeup, hairstyle, and fashion suggestions.

[0680] Input: Feature data, user preferences and trend information

[0681] Output: Generated proposal data

[0682] Step 8:

[0683] The server converts the generated proposal data into JSON format and sends it to the device using a REST API, sending the encoded data packet to the device.

[0684] Input: Generated proposal data

[0685] Output: Proposal data in JSON format, data sent to the device

[0686] Step 9:

[0687] The device displays the received recommendation data to the user, who then checks the displayed recommendations and selects their preferred beauty salon, aesthetic salon, or fashion store.

[0688] Input: JSON formatted proposal data received from the server

[0689] Output: The suggestions displayed to the user

[0690] Step 10:

[0691] The user follows the suggestions, enters the necessary information (service details, date and time, etc.) into the reservation form within the application, and presses the reservation button to confirm the reservation.

[0692] Input: Booking information entered by the user

[0693] Output: Confirmed reservation information

[0694] Step 11:

[0695] The terminal sends the confirmed reservation information to the server, and the server notifies the store of the received reservation information.

[0696] Input: Reservation information confirmed by the user

[0697] Output: Reservation information sent to the server, reservation information notified to the target store

[0698] Step 12:

[0699] To make a payment using the payment platform, the user inputs the necessary payment information (such as card information) into the payment interface on the terminal, which then transmits this information to the payment platform for payment processing.

[0700] Input: User's payment information

[0701] Output: Payment information sent to the payment platform, payment processing results

[0702] Step 13:

[0703] The server receives a payment completion notification from the payment platform and transmits the information to the terminal together with the user's reservation information.

[0704] Input: Payment completion notification from payment platform

[0705] Output: Payment completion notification and reservation information sent to the user's terminal

[0706] Through these processing steps, users can receive personalized beauty and fashion recommendations using generative AI models, and make reservations and payments smoothly.

[0707] (Application example 1)

[0708] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0709] Systems that offer beauty and fashion recommendations are expected to enable users to quickly and accurately receive personalized recommendations, while also enabling smooth in-store reservations and payment. While conventional systems analyze users' facial photos to provide recommendations, these recommendations often do not adequately address individual preferences or trends. Furthermore, the reservation and payment processes are complicated, resulting in poor user convenience. Therefore, there is a need for a system that allows users to receive personalized recommendations and easily make reservations and payments.

[0710] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0711] In this invention, the server includes means for a user to upload a facial photo, means for transmitting the facial photo to the server, means for the server to analyze the facial photo using a generative AI model, means for suggesting makeup, hairstyles, and fashions that suit the user's preferences based on the analysis results obtained by the generative AI model, means for making a store reservation based on the suggested makeup, hairstyles, and fashions, means for making a payment through a payment platform, means for saving the user's data in an internal database, and interface means for checking the suggestion results on a smartphone. This allows the user to receive personalized beauty and fashion suggestions and smoothly make a reservation and make a payment at the suggested store.

[0712] "Means for users to upload facial photos" refers to the interface and functionality that allows users to select or take a photo of their face using a smartphone or other device and upload it into the application.

[0713] The "means for transmitting facial photographs to a server" refers to a communication protocol and interface for securely and efficiently transmitting uploaded facial photographs of users to a server.

[0714] "Means for the server to analyze facial photos using a generative AI model" refers to the function of inputting facial photo data received by the server into a generative AI model, which then analyzes facial features and extracts them as data.

[0715] "A means of suggesting makeup, hairstyles, and fashion that suit the user's preferences based on the analysis results obtained by a generative AI model" is a system that uses the results of facial photo analysis by a generative AI model to suggest optimal beauty and fashion options in combination with the user's preference information.

[0716] "Means for making store reservations based on suggested makeup, hairstyles, and fashion" refers to the interface and functionality for making reservations at hair salons, beauty salons, and fashion stores through the application based on the suggested results of the generative AI model.

[0717] "Means for making payments through a payment platform" means a payment system and interface for completing online payments for the booked services.

[0718] "Means for storing user data in an internal database" refers to a database system for securely storing user facial photographs and preference data for use in subsequent suggestions.

[0719] The "interface means for checking the proposal results on a smartphone" refers to a user interface that allows a user to visually check the beauty and fashion proposals from the generated AI model using a smartphone.

[0720] This invention is a system that allows users to receive personalized beauty and fashion suggestions, and smoothly make store reservations and payments. This system includes the following means.

[0721] A way for users to upload photos of their faces

[0722] The user starts the smartphone application, takes a photo of their face using the camera function or selects an existing photo from the gallery, and then uploads the photo data to the application through an interface for uploading a photo of their face.

[0723] A means of sending a face photo to the server

[0724] The device temporarily stores the uploaded facial photo data and prepares it for transmission to the server. The stored facial photo data is then sent to the server as an HTTP request. This is mainly done using the React Native HTTP request module.

[0725] The server generates a facial photo and analyzes it using an AI model.

[0726] The server inputs the facial photo data received from the device into the generative AI model. At this time, the server uses TensorFlow and PyTorch to analyze the facial photo's features (e.g., contours, skin tone, eye shape, etc.) and converts them into numerical data. The analysis results are then stored in an internal database (MongoDB) and transferred for subsequent processing.

[0727] How generative AI models generate proposals

[0728] The server retrieves the results of the user's facial photo analysis from an internal database, inputs the user's preferences and current trend information into the AI ​​model, and the generative AI model generates optimal suggestions for makeup, hairstyles, and fashion, and sends the generated suggestions in JSON format to the device.

[0729] A way to make restaurant reservations based on the proposed content

[0730] The user checks the suggestions and selects the hair salon, aesthetic salon, or fashion store that they like. They then enter reservation information, such as the service content and date and time of the selected store, and confirm the reservation.

[0731] A means of making payments through a payment platform

[0732] The terminal sends the reservation information entered by the user to the server. It then displays an interface for entering payment information, and the user enters the payment information. The entered payment information is sent to a payment platform (e.g., Stripe API) and the processing result is received. The server notifies the store of the reservation information received from the terminal, receives a payment completion notification from the payment platform, and confirms the completion of the reservation. The completed reservation and payment information are sent to the terminal.

[0733] Specific example explanation

[0734] For example, a user launches the "Beauty360" app on their smartphone and uploads a photo of their face. They then enter "I'm interested in natural makeup and bob hairstyles" through the interface. The server analyzes the photo and uses this information to generate optimal recommendations using a generative AI model. The user then confirms the recommendations, makes a reservation at the suggested salon, and makes payment via Stripe.

[0735] Example prompt sentence:

[0736] {

[0737] "face_features": {

[0738] "face_shape": "round",

[0739] "skin_tone": "fair",

[0740] "eye_shape": "almond"

[0741] },

[0742] "user_preference": {

[0743] "style": "natural",

[0744] "interest": ["hair", "makeup"]

[0745] },

[0746] "current_trends": {

[0747] "makeup": ["minimalist", "soft colors"],

[0748] "hair": ["long bob", "wavy"]

[0749] }

[0750] }

[0751] By writing and implementing it in this way, users will be able to receive personalized beauty and fashion suggestions, and easily make reservations and payments at physical stores using a smartphone application.

[0752] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0753] Step 1:

[0754] A user starts the smartphone application and takes a face photo using the camera function or selects an existing face photo from the gallery. This inputs face photo data. Next, the face photo data is uploaded to the application through the face photo upload interface and temporarily saved on the device.

[0755] Step 2:

[0756] The device sends the temporarily saved facial photo data to the server as an HTTP request. The server receives the facial photo data and prepares to start processing it as input data.

[0757] Step 3:

[0758] After receiving the facial photo data, the server uses TensorFlow to input the facial photo into a generative AI model. The generative AI model analyzes the facial photo's features (contours, skin tone, eye shape, etc.) and converts these features into numerical data. The converted data is stored in an internal database (e.g., MongoDB) and transferred for subsequent processing.

[0759] Step 4:

[0760] The server retrieves the results of the user's facial photo analysis from its internal database and inputs them into the generative AI model along with the user's preferences and trend information. The generative AI model then generates makeup, hairstyle, and fashion suggestions and outputs them in JSON format. The output suggestions are then sent to the device.

[0761] Step 5:

[0762] The user checks the recommendations through their smartphone application. At this time, the user interface visually displays the beauty recommendations created by the generative AI model. The user then selects their favorite beauty salon, beauty salon, or fashion store.

[0763] Step 6:

[0764] The user inputs the service details and date and time of the selected store and sends the reservation information to the server. The server notifies the corresponding store of the received reservation information. The notified store confirms the reservation and sends the reservation confirmation information to the server.

[0765] Step 7:

[0766] The terminal sends the reservation information entered by the user to the server, and then displays an interface for entering payment information. The user enters the payment information, and the terminal sends the information to a payment platform (e.g., Stripe API) for processing. The processing result is sent to the server.

[0767] Step 8:

[0768] The server receives a payment completion notification from the payment platform and confirms the completion of the reservation. This completion information is sent to the terminal so that the user can check it. This completes the entire process.

[0769] By following the above steps, users can receive personalized beauty and fashion recommendations and smoothly make reservations and payments. Examples of prompts are as follows:

[0770] {

[0771] "face_features": {

[0772] "face_shape": "round",

[0773] "skin_tone": "fair",

[0774] "eye_shape": "almond"

[0775] },

[0776] "user_preference": {

[0777] "style": "natural",

[0778] "interest": ["hair", "makeup"]

[0779] },

[0780] "current_trends": {

[0781] "makeup": ["minimalist", "soft colors"],

[0782] "hair": ["long bob", "wavy"]

[0783] }

[0784] }

[0785] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0786] The present invention is a system that provides beauty and fashion recommendations based on a user's facial photo and preferences, and allows for smooth reservations and payments. It also has the ability to recognize the user's emotional state and make optimal recommendations based on that. The following elements are required to implement this system:

[0787] A way for users to upload photos of their faces

[0788] User:

[0789] 1. Launch the application and take a photo of your face using the camera function or select an existing photo from your gallery.

[0790] 2. Upload your face photo data to the application through the interface for uploading face photos.

[0791] A means of sending a face photo to the server

[0792] Device:

[0793] 1. The uploaded facial photo data is temporarily saved and prepared for sending to the server.

[0794] 2. The saved facial photo data is sent to the server as an HTTP request.

[0795] The server generates a facial photo and analyzes it using an AI model.

[0796] server:

[0797] 1. The facial photo data received from the device is input into the generative AI model.

[0798] 2. The generative AI model analyzes the facial features (e.g., facial contours, skin tone, eye shape, etc.) and converts them into numerical data.

[0799] 3. The analysis results are stored in an internal database for subsequent processing.

[0800] Using an emotion engine to recognize user emotions

[0801] server:

[0802] 1. Input facial photo data into the emotion engine to analyze the user's emotional state (e.g., happiness, sadness, surprise, etc.).

[0803] 2. The analysis results of the emotion engine are stored in an internal database.

[0804] How generative AI models generate proposals

[0805] server:

[0806] 1. Obtain the analysis results and emotional state of the user's face photo from the internal database.

[0807] 2. User-input preferences and current trend information are fed into the AI ​​model to generate optimal makeup, hairstyle, and fashion suggestions.

[0808] 3. It also dynamically adjusts its suggestions based on your emotional state.

[0809] 4. The generated proposal results are sent to the device.

[0810] A way to make a reservation based on the suggestions

[0811] User:

[0812] 1. Check the suggestions and select your favorite hair salon, beauty salon, or fashion store.

[0813] 2. Enter reservation information such as the service content, date and time of the selected store and confirm your reservation.

[0814] A means of making payments through a payment platform

[0815] Device:

[0816] 1. The reservation information entered by the user is sent to the server.

[0817] 2. An interface for entering payment information is displayed, and the user enters the payment information.

[0818] 3. The entered payment information is sent to the payment platform and the processing result is received.

[0819] server:

[0820] 1. Notify beauty salons, esthetic salons, and fashion stores of reservation information received from the device.

[0821] 2. Receive a payment confirmation from the payment platform to confirm the completion of your booking.

[0822] 3. Send the completed booking and payment information to the terminal.

[0823] Specific examples

[0824] Example 1: Makeup and hairstyle suggestions

[0825] User:

[0826] 1. Upload a photo of your face using the application and indicate that you are interested in natural makeup and bob hairstyles.

[0827] 2. Check the suggestions from the server and make a reservation at the suggested beauty salon.

[0828] Device:

[0829] 1. Send a face photo and user preferences to the server.

[0830] 2. Receive suggestions from the server and present them to the user.

[0831] 3. Send the reservation information to the server and display the payment interface.

[0832] server:

[0833] 1. Analyze your facial photo and suggest the best makeup and hairstyle.

[0834] 2. The emotion engine analyzes the user's emotional state and adjusts the suggestions accordingly.

[0835] 3. Notify the beauty salon of the user's reservation information and receive payment completion information.

[0836] Example 2: Fashion coordination suggestions

[0837] User:

[0838] 1. Enter your interest in casual style and upload a photo of your face.

[0839] 2. Make an appointment at the suggested fashion store and complete the payment online.

[0840] Device:

[0841] 1. Send your face photo and preference information to the server.

[0842] 2. Receive suggestions from the server and present them to the user.

[0843] 3. Send the reservation information to the server and process the payment.

[0844] server:

[0845] 1. We suggest the best fashion coordination based on the analysis of your face photo and your preferences.

[0846] 2. The emotion engine analyzes the user's emotional state and adjusts the suggestions accordingly.

[0847] 3. The user's reservation information is notified to the fashion store and payment completion information is received.

[0848] As described above, the system of the present invention allows users to receive personalized beauty and fashion suggestions using generative AI models and emotion engines, and to easily make reservations and payments.

[0849] The processing flow will be explained below.

[0850] Step 1:

[0851] User: Launches the application and takes a photo of his / her face using the camera function or selects an existing photo from the gallery. Clicks the button to upload a photo of his / her face.

[0852] Step 2:

[0853] Terminal: The facial photo data selected or taken by the user is temporarily stored within the application. An HTTP request is generated to send the stored facial photo data to the server. The generated HTTP request is sent to the server.

[0854] Step 3:

[0855] Server: Receives facial photo data sent from the device. Preprocesses the facial photo data to pass it to the generative AI model and emotion engine.

[0856] Step 4:

[0857] Server: The generative AI model analyzes the facial features (e.g., facial contours, skin tone, eye shape, etc.) and converts them into numerical data. The analysis results are stored in an internal database.

[0858] Step 5:

[0859] Server: Inputs a facial photo into the emotion engine and analyzes the user's emotional state (e.g., happiness, sadness, surprise, etc.). The analysis results of the emotion engine are also stored in an internal database.

[0860] Step 6:

[0861] User: Fills out a form to select their makeup, hairstyle and fashion preferences; answers choices and questions about current trends; completes the form and clicks the "Submit" button.

[0862] Step 7:

[0863] Terminal: The application temporarily stores the preferences and trend information entered by the user. It generates an HTTP request to send the stored data to the server. It then sends the generated HTTP request to the server.

[0864] Step 8:

[0865] Server: Receives preferences and trend information sent by the user. Combining the results of facial photo analysis, emotional state, and preference information, the generative AI model generates optimal makeup, hairstyle, and fashion suggestions. The suggestions are dynamically adjusted based on the user's emotional state.

[0866] Step 9:

[0867] Server: Sends the proposed results from the generative AI model to the terminal.

[0868] Step 10:

[0869] Terminal: Receives the proposal results sent from the server. The received proposal results are displayed within the application and confirmed by the user.

[0870] Step 11:

[0871] User: Check the suggestions and select a hair salon, beauty salon, or fashion store that suits them best. Enter the reservation information (date, time, service details, etc.) for the selected store and click the "Confirm reservation" button.

[0872] Step 12:

[0873] Terminal: The reservation information entered by the user is temporarily saved within the application. An HTTP request is generated to send the reservation information to the server. The generated HTTP request is sent to the server.

[0874] Step 13:

[0875] Server: Receives reservation information sent from the terminal. Generates and sends another HTTP request to the store's system to notify the corresponding store of the received reservation information. Receives reservation confirmation or adjustment information from the store and stores it.

[0876] Step 14:

[0877] Server: Sends reservation confirmation information to the terminal.

[0878] Step 15:

[0879] Terminal: Receives the reservation confirmation information sent from the server and displays it to the user. Generates a link to the payment platform and presents it to the user.

[0880] Step 16:

[0881] User: Click on the payment platform link displayed on the terminal, select the payment method, enter the required payment information, and click the "Complete Payment" button.

[0882] Step 17:

[0883] Terminal: Sends the payment information entered by the user to the payment platform. Receives the payment completion notification and generates an HTTP request to send to the server. Sends the generated HTTP request to the server.

[0884] Step 18:

[0885] Server: Receives the payment completion notification sent from the terminal. Based on the received information, sends a reservation completion and payment completion notification to the relevant store. Sends a completion notification to the terminal.

[0886] Step 19:

[0887] Terminal: Receives the reservation and payment completion notification sent from the server and displays it to the user.

[0888] Example 2

[0889] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0890] Conventional beauty and fashion recommendation systems have difficulty providing personalized recommendations that comprehensively consider a user's facial photo, emotional state, preferences, etc. Furthermore, there has been a lack of systems that allow users to instantly complete reservations and payments based on the recommendations. Therefore, there has been a strong demand for the development of a system that allows users to easily receive optimal beauty and fashion recommendations and quickly use the service.

[0891] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0892] In this invention, the server includes means for using an emotion engine that analyzes the user's emotional state, means for adjusting the content of suggestions according to the emotional state, and means for suggesting makeup, hairstyles, and fashions that suit the user's preferences based on the analysis results obtained by the generative AI model. This makes it possible to quickly provide personalized and optimal suggestions by comprehensively considering the user's facial photo and emotional state, and to smoothly execute the reservation and payment processes.

[0893] A "user" is a person who uses the system to upload a photo of their face and receive beauty and fashion suggestions.

[0894] A "face photo" is still image data that includes features such as the user's facial contours, eye shape, and skin tone.

[0895] The "server" is a central processing unit that processes facial photo data, user preferences, and emotional state, and generates optimal suggestions using generative AI models and emotion engines.

[0896] A "generative AI model" is an artificial intelligence model that analyzes facial photos and converts their features into numerical data.

[0897] An "emotion engine" is software that analyzes a user's emotional state (e.g., happiness, sadness, surprise, etc.) from a photograph of their face.

[0898] "Makeup" refers to methods and products for applying makeup to a user's face.

[0899] A "hairstyle" is a method or design for instructing the shape and arrangement of a user's hair.

[0900] "Fashion" refers to a user's clothing, accessories, and other decorative items related to their appearance.

[0901] A "reservation" is a procedure in which a user determines in advance the date, time, and content of services to be used at a beauty salon, esthetic salon, or fashion store.

[0902] "Payment Platform" means an online payment system through which Users pay for Services.

[0903] "Suggestions" are specific advice and options regarding makeup, hairstyles, and fashion provided to users based on the results of analysis by generative AI models and emotion engines.

[0904] MODE FOR CARRYING OUT THE INVENTION

[0905] This system provides beauty and fashion recommendations based on a user's facial photograph and preferences, and allows for smooth booking and payment. It also has the ability to recognize the user's emotional state and provide optimal recommendations based on that. The specific configuration and operating procedures for implementing this system are described below.

[0906] A way for users to upload photos of their faces

[0907] The user launches the application on their smartphone or PC and takes a photo of their face using the camera function, or selects an existing photo from the gallery. Then, the user presses the upload button on the application to upload the photo data to the application. This application can run on the iOS or Android platform, for example.

[0908] A means of sending a face photo to the server

[0909] The device temporarily stores the uploaded facial photo data and sends it to the server in the form of an HTTP request. The HTTP request header contains authentication information, and the body contains the facial photo data. The device's local storage is used for temporary storage.

[0910] The server generates a facial photo and analyzes it using an AI model.

[0911] The server inputs the facial photo data received from the device into a generative AI model. The server performs preprocessing on the facial photo data, such as resizing and color normalization. The data is then input into a specialized facial analysis AI model (e.g., a convolutional neural network using TensorFlow) to convert features such as facial contours, skin tone, and eye shape into numerical data. The results of this analysis are stored in an internal database and used for subsequent processing.

[0912] Using an emotion engine to recognize user emotions

[0913] The server inputs the facial photo data into an emotion engine to analyze the user's emotional state (e.g., happiness, sadness, surprise, etc.). The emotion engine's analysis results are also stored in an internal database. The emotion engine uses, for example, facial expression recognition software (e.g., Microsoft's Azure Emotion API).

[0914] How generative AI models generate proposals

[0915] The server retrieves the analysis results of the user's facial photo and emotional state from an internal database, inputs the user's preferences and current trend information into the AI ​​model, and generates optimal suggestions for makeup, hairstyles, and fashion. Furthermore, the suggestions are dynamically adjusted according to the user's emotional state. The results of these suggestions are returned to the device in JSON format.

[0916] For example, if a user enters "I'm interested in natural makeup and bob hairstyles" and uploads a photo of their face, the server will use a generative AI model to analyze the user's facial features and make optimal suggestions for natural makeup and bob hairstyles based on those features.

[0917] Example prompt sentence:

[0918] "I'm interested in natural makeup and bob hairstyles."

[0919] A way to make a reservation based on the suggestions

[0920] The user reviews the suggestions and selects their favorite beauty salon, esthetic salon, or fashion store on the application. They then enter reservation information such as the service content and date and time of the selected store and press "Confirm reservation" to make the reservation. The reservation information is then sent from the device to the server.

[0921] A means of making payments through a payment platform

[0922] The terminal sends the reservation information entered by the user to the server, and also displays an interface for entering payment information, where the user enters payment information such as credit card information. The terminal sends data to the API of the payment platform (e.g., Stripe or PayPal) and receives the processing results.

[0923] A means by which the server processes booking and payment information

[0924] The server notifies the beauty salon, esthetic salon, or fashion store of the reservation information received from the device. It also receives a payment completion notification from the payment platform and confirms the completion of the reservation. The completed reservation and payment information are returned to the device in JSON format.

[0925] As described above, the system of the present invention allows users to receive personalized beauty and fashion suggestions using generative AI models and emotion engines, and to easily make reservations and payments.

[0926] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0927] Step 1:

[0928] Users launch the application on their smartphone or PC, take a photo of their face using the camera function, or select an existing photo from the gallery, and then press the upload button to upload the photo data to the application.

[0929] Input: Facial photo data taken or selected by the user

[0930] Output: Facial photo data sent to the application through the upload interface

[0931] Specific actions: The user taps the app's camera icon to launch the camera, takes a photo of their face, and taps "Done." Alternatively, the user taps the gallery icon, selects an existing photo, and presses the "Upload" button.

[0932] Step 2:

[0933] The device temporarily stores the uploaded facial photo data and sends it to the server in the form of an HTTP request.

[0934] Input: Uploaded face photo data

[0935] Output: Facial photo data sent to the server in the form of an HTTP request

[0936] Specific operation: The device temporarily stores the face photo data in local storage and then sends it to the server's API endpoint in the background. The HTTP request header contains authentication information, and the body contains the face photo data.

[0937] Step 3:

[0938] The server inputs the facial photo data received from the device into the generative AI model. The server preprocesses the facial photo data by resizing and color normalizing it, and then inputs the data into the generative AI model.

[0939] Input: Facial photo data received from the device

[0940] Output: Analysis results converted from facial features into numerical data

[0941] Specific operation: The server performs preprocessing on the facial photo data, such as resizing and color normalizing, and then inputs the data into a generative AI model (e.g., a convolutional neural network using TensorFlow), converting features such as facial contours, skin tone, and eye shape into numerical data.

[0942] Step 4:

[0943] The server inputs the facial photo data into an emotion engine to analyze the user's emotional state (e.g., happiness, sadness, surprise, etc.).

[0944] Input: Facial photo data

[0945] Output: Analysis results showing the user's emotional state

[0946] Specific operation: The server inputs facial photo data into an emotion engine (e.g., facial expression recognition software), analyzes the emotional state, and stores the results in an internal database.

[0947] Step 5:

[0948] The server retrieves the analysis results of the user's facial photo and emotional state from an internal database, and inputs the user's input preferences and current trend information into an AI model to generate optimal suggestions for makeup, hairstyles, and fashion.

[0949] Input: User's facial photo analysis results, emotional state, preferences and trend information

[0950] Output: Makeup, hairstyle, and fashion suggestions

[0951] Specific operation: The server retrieves the necessary data from the internal database, integrates them, and inputs them into the generative AI model. The generative AI model generates a proposal, adjusts the proposal content using the emotion engine, and sends it to the device in JSON format.

[0952] Step 6:

[0953] The user checks the suggestions on the application and selects the hair salon, beauty salon, or fashion store that suits them best. After that, they enter reservation information such as the service content, date and time, and confirm the reservation.

[0954] Input: Proposal details, reservation information (service details, date and time)

[0955] Output: Confirmed reservation information

[0956] Specific operation: The user checks the list of suggestions, taps the selection button to move to the reservation page, enters the service details and date and time, and taps "Confirm reservation."

[0957] Step 7:

[0958] The terminal transmits the reservation information entered by the user to the server, displays an interface for entering payment information, and the user enters the payment information. The terminal then transmits the data to the payment platform and receives the processing result.

[0959] Input: Reservation information, payment information

[0960] Output: Processing result by payment platform

[0961] Specific operation: The terminal sends the reservation information to the server, then displays the payment page. The user enters credit card information and taps the "Pay" button. The terminal then sends the data to the payment platform's API and receives the processing result.

[0962] Step 8:

[0963] The server notifies the beauty salon, esthetic salon, or fashion store of the reservation information received from the terminal, receives a payment completion notification from the payment platform, confirms the completion of the reservation, and sends the completed reservation and payment information to the terminal.

[0964] Input: Reservation information, payment completion notification

[0965] Output: Reservation completion information, payment completion information

[0966] Specific operation: The server notifies the store management system of the reservation information in real time, then receives a payment completion notification from the payment platform, updates the reservation status to "Completed", and returns the reservation and payment completion information to the terminal in JSON format.

[0967] (Application example 2)

[0968] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0969] Conventional beauty and fashion recommendation systems were unable to provide personalized recommendations that fully considered the individual characteristics and emotional state of each user. As a result, users were often dissatisfied with the recommendations and stopped using the service. In addition, the reservation and payment procedures were cumbersome, leaving a need for an improved user experience.

[0970] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0971] In this invention, the server includes means for a user to upload a facial photo, means for transmitting the facial photo to the server, means for the server to use a generative AI model that analyzes the facial photo, means for suggesting beauty and fashion that matches the user's preferences based on the analysis results obtained by the generative AI model, means for making a reservation at a service facility based on the suggested beauty and fashion, means for making a payment through a payment platform, means for inputting the user's facial photo and preference information through a smartphone application, means for recognizing the user's emotional state using an emotion engine, and means for dynamically adjusting the content of the proposals according to the emotional state. This not only enables personalized proposals based on the user's individual characteristics and emotional state, but also enables a smooth process from reservation to payment.

[0972] "Means for users to upload facial photos" refers to the methods or functions that allow users to take or select their own facial photos and transmit the photo data via the Internet.

[0973] "Means for sending facial photos to a server" refers to a system that has the function of temporarily storing facial photos taken or selected by a user and sending them to an external server via the Internet.

[0974] "Means of using generative AI models" refers to methods or functions that use models to generate results by using artificial intelligence to analyze collected data.

[0975] "Means of suggesting beauty and fashion that suits the user's preferences" refers to a function that suggests beauty, hairstyles, and fashion that suits the user's individual preferences based on the analysis results obtained by the generative AI model.

[0976] "Means for making reservations at service providing facilities" refers to a function that enables users to make online reservations at service providing facilities such as beauty salons, esthetic salons, and fashion stores based on the beauty and fashion recommendations.

[0977] "Means of making payments through a payment platform" refers to the use of payment systems and services to make payments for online reservations in a secure and efficient manner.

[0978] "Means for inputting a user's facial photograph and preferred information via a smartphone application" refers to an interface that allows a user to input a photograph of their face and preferred beauty and fashion information via an application that runs on a smartphone.

[0979] "Means for recognizing a user's emotional state using an emotion engine" refers to a method that uses a system or algorithm to analyze a facial photograph and determine the user's current emotional state (such as joy, sadness, surprise, etc.).

[0980] "Means for dynamically adjusting the content of suggestions according to the emotional state" refers to a function that appropriately changes and adjusts the content of beauty and fashion suggestions in real time based on the recognized emotional state of the user.

[0981] The present invention is a system in which a user uploads a photo of their face via a smartphone application, and the server analyzes the photo using a generated AI model to suggest beauty and fashion products that match the user's preferences. This system is realized through the following configuration and processing.

[0982] The server first receives a facial photo taken or selected by the user using a smartphone application. The facial photo is uploaded using the smartphone's camera or gallery function. This photo data is then sent to the server via the Internet.

[0983] The server temporarily stores the received facial photo data and analyzes it using a generative AI model. The analysis extracts facial features such as facial contours, skin tone, and eye shape as numerical data. This process uses machine learning frameworks such as TensorFlow.

[0984] Furthermore, the server uses an emotion engine to recognize the user's emotional state. The emotion engine can determine the user's current emotion (e.g., joy, sadness, surprise, etc.) from the facial photograph. This information is also stored in an internal database as numerical data.

[0985] The server uses a generative AI model to generate optimal beauty and fashion recommendations based on the user's preferences, current trends, and the acquired facial features and emotional state. The recommendations are dynamically adjusted according to the user's emotional state and presented in a form optimized for each individual user. The recommendations are displayed to the user via a smartphone application.

[0986] Users can check the suggestions and make reservations at beauty salons, esthetic salons, and fashion stores that they are interested in. They enter reservation information through the smartphone application and confirm the reservation. The reservation information is sent to the server, which then notifies the relevant service provider.

[0987] Finally, payment is processed. The smartphone application connects to a payment platform and securely processes the payment information entered by the user. Services such as Stripe can be used as payment platforms. A notification of payment completion is sent to the server, confirming the completion of the reservation.

[0988] As a concrete example, consider the following prompt sentence:

[0989] Example prompt sentence:

[0990] "User's face photo has been uploaded and preprocessed. Generate a personalized beauty or fashion suggestion based on the following parameters:

[0991] Face features: [numerical data such as face shape, skin tone, eye shape, etc.]

[0992] Emotion state: "Joy"

[0993] User preferences: ["Natural makeup", "Bob hairstyle"]

[0994] Current trends: ["Spring 2023 trends"]

[0995] This will enable users to receive beauty and fashion suggestions based on their facial photos, preferences, and real-time emotional state, and will also enable them to complete reservations and payments on the spot.

[0996] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0997] Step 1:

[0998] The user starts the smartphone application and takes a photo of their face or selects one from the gallery. They then use the application's camera or gallery function to obtain a photo of their face and press the upload button to enter the photo data. The entered photo data is temporarily saved.

[0999] Step 2:

[1000] The device sends face photo data to the server. The face photo data is temporarily saved and then sent to the server via an HTTP request. The input in this step is the face photo data, and the output is the data sent to the server.

[1001] Step 3:

[1002] The server inputs the received facial photo data into a generative AI model for analysis. Using a machine learning framework such as TensorFlow, facial photo features (contours, skin tone, eye shape, etc.) are extracted as numerical data. The input is facial photo data, and the output is numerical data indicating facial features.

[1003] Step 4:

[1004] The server uses an emotion engine to recognize the user's emotional state. The received facial photo data is input into the emotion engine to determine the user's emotional state (e.g., joy, sadness, surprise, etc.). The emotion engine's specific operation uses an algorithm to analyze facial expressions and micro-expressions. The input is facial photo data, and the output is data indicating the user's emotional state.

[1005] Step 5:

[1006] The server generates beauty and fashion suggestions using a generative AI model based on facial feature data, emotional state data, user preference information, and trend information. This uses prompts that take into account the user's input preferences (e.g., natural makeup, bob hairstyle) and current trend information. Specific examples of generated prompts include:

[1007] "User's face photo has been uploaded and preprocessed. Generate a personalized beauty or fashion suggestion based on the following parameters:

[1008] Face features: [numerical data such as face shape, skin tone, eye shape, etc.]

[1009] Emotion state: "Joy"

[1010] User preferences: ["Natural makeup", "Bob hairstyle"]

[1011] Current trends: ["Spring 2023 trends"]

[1012] The inputs are facial feature data, emotional state data, user preference information, and trend information, and the output is suggested beauty and fashion content.

[1013] Step 6:

[1014] The user checks the proposed beauty and fashion details on a smartphone application and selects the service provider of interest (beauty salon, esthetic salon, fashion store). The user then enters the reservation information for the desired service (date, time, location, service details, etc.) and presses the reservation button. The input is the confirmation result of the proposal details and the reservation information, and the output is the entered reservation information.

[1015] Step 7:

[1016] The terminal sends the entered reservation information to the server, which then notifies the relevant service provider of that information. The input is the reservation information, and the output is a notification to the service provider.

[1017] Step 8:

[1018] A user enters payment information into a smartphone application to make a payment. The application then sends the entered payment information to a payment platform (e.g., Stripe) to process the payment. The input is the payment information, and the output is the payment processing result.

[1019] Step 9:

[1020] The server receives the payment completion notification and confirms the completion of the reservation. It then notifies the user of the completed reservation information and payment information. The input is the payment processing result, and the output is the completed reservation information and payment completion notification.

[1021] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1022] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1023] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[1024] [Third embodiment]

[1025] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[1026] 5, the data processing system 310 includes the data processing device 12 and a headset type terminal 314. An example of the data processing device 12 is a server.

[1027] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1028] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[1029] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1030] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1031] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1032] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1033] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1034] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1035] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1036] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[1037] The present invention is a system that allows users to receive personalized beauty and fashion recommendations and smoothly make reservations and payments. The following elements are required to implement this system:

[1038] A way for users to upload photos of their faces

[1039] User:

[1040] 1. Launch the application and take a photo of your face using the camera function or select an existing photo from your gallery.

[1041] 2. Upload the facial photo data to the application through the interface for uploading facial photos.

[1042] A means of sending a face photo to the server

[1043] Device:

[1044] 1. The uploaded facial photo data is temporarily saved and prepared for sending to the server.

[1045] 2. The saved facial photo data is sent to the server as an HTTP request.

[1046] The server generates a facial photo and analyzes it using an AI model.

[1047] server:

[1048] 1. The facial photo data received from the device is input into the generative AI model.

[1049] 2. The generative AI model analyzes the facial features (e.g., facial contours, skin tone, eye shape, etc.) and converts them into numerical data.

[1050] 3. Store the analysis results in an internal database and forward them for further processing.

[1051] How generative AI models generate proposals

[1052] server:

[1053] 1. Obtain the analysis results of the user's facial photo from the internal database.

[1054] 2. User-input preferences and current trend information are fed into the AI ​​model to generate optimal makeup, hairstyle, and fashion suggestions.

[1055] 3. The generated proposal results are sent to the device in JSON format.

[1056] A way to make a reservation based on the suggestions

[1057] User:

[1058] 1. Check the suggestions and select your favorite hair salon, beauty salon, or fashion store.

[1059] 2. Enter reservation information such as the service content, date and time of the selected store and confirm your reservation.

[1060] A means of making payments through a payment platform

[1061] Device:

[1062] 1. The reservation information entered by the user is sent to the server.

[1063] 2. An interface for entering payment information is displayed, and the user enters the payment information.

[1064] 3. The entered payment information is sent to the payment platform and the processing result is received.

[1065] server:

[1066] 1. Notify beauty salons, esthetic salons, and fashion stores of reservation information received from the device.

[1067] 2. Receive a payment confirmation from the payment platform to confirm the completion of your booking.

[1068] 3. Send the completed booking and payment information to the terminal.

[1069] Specific examples

[1070] Example 1: Makeup and hairstyle suggestions

[1071] User:

[1072] 1. Upload a photo of your face using the application and indicate that you are interested in natural makeup and bob hairstyles.

[1073] 2. Check the suggestions from the server and make a reservation at the suggested beauty salon.

[1074] Device:

[1075] 1. Send a face photo and user preferences to the server.

[1076] 2. Receive suggestions from the server and present them to the user.

[1077] 3. Send the reservation information to the server and display the payment interface.

[1078] server:

[1079] 1. Analyze your facial photo and suggest the best makeup and hairstyle.

[1080] 2. Notify the beauty salon of the user's reservation information and receive payment completion information.

[1081] Example 2: Fashion coordination suggestions

[1082] User:

[1083] 1. Enter your interest in casual style and upload a photo of your face.

[1084] 2. Make an appointment at the suggested fashion store and complete the payment online.

[1085] Device:

[1086] 1. Send your face photo and preference information to the server.

[1087] 2. Receive suggestions from the server and present them to the user.

[1088] 3. Send the reservation information to the server and process the payment.

[1089] server:

[1090] 1. We suggest the best fashion coordination based on the analysis of your face photo and your preferences.

[1091] 2. The user's reservation information is notified to the fashion store and payment completion information is received.

[1092] As described above, the system of the present invention allows users to receive personalized beauty and fashion suggestions using generative AI models, and easily make reservations and payments.

[1093] The processing flow will be explained below.

[1094] Step 1:

[1095] User: Launches the application and takes a photo of his / her face using the camera function or selects an existing photo. Clicks the button to upload a photo of his / her face.

[1096] Step 2:

[1097] Terminal: The facial photo data selected or taken by the user is temporarily stored within the application. An HTTP request is generated to send the stored facial photo data to the server. The generated HTTP request is sent to the server.

[1098] Step 3:

[1099] Server: Receives facial photo data sent from the device. The facial photo data is input into a generative AI model to analyze the user's facial features, contours, skin tone, etc. The analysis results are temporarily stored and used for subsequent processing.

[1100] Step 4:

[1101] User: Fills out a form to select their makeup, hairstyle and fashion preferences; answers choices and questions about current trends; completes the form and clicks the "Submit" button.

[1102] Step 5:

[1103] Terminal: The application temporarily stores the preferences and trend information entered by the user. It generates an HTTP request to send the stored data to the server. It then sends the generated HTTP request to the server.

[1104] Step 6:

[1105] Server: Receives preferences and trend information sent by the user. Combining the facial photo analysis results with preference information, the generative AI model generates optimal makeup, hairstyle, and fashion suggestions. The generated suggestions are temporarily stored.

[1106] Step 7:

[1107] Server: Sends the proposed results from the generative AI model to the terminal.

[1108] Step 8:

[1109] Terminal: Receives the proposal results sent from the server. The received proposal results are displayed within the application and confirmed by the user.

[1110] Step 9:

[1111] User: Check the suggestions and select a hair salon, beauty salon, or fashion store that suits them best. Enter the reservation information (date, time, service details, etc.) for the selected store and click the "Confirm reservation" button.

[1112] Step 10:

[1113] Terminal: The reservation information entered by the user is temporarily saved within the application. An HTTP request is generated to send the saved reservation information to the server. The generated HTTP request is sent to the server.

[1114] Step 11:

[1115] Server: Receives reservation information sent from the terminal. Generates and sends another HTTP request to the store's system to notify the corresponding store of the received reservation information. Receives reservation confirmation or adjustment information from the store and stores it.

[1116] Step 12:

[1117] Server: Sends reservation confirmation information to the terminal.

[1118] Step 13:

[1119] Terminal: Receives the reservation confirmation information sent from the server and displays it to the user. Generates a link to the payment platform and presents it to the user.

[1120] Step 14:

[1121] User: Click on the payment platform link displayed on the terminal, select the payment method, enter the required payment information, and click the "Complete Payment" button.

[1122] Step 15:

[1123] Terminal: Sends the payment information entered by the user to the payment platform. Receives the payment completion notification and generates an HTTP request to send to the server. Sends the generated HTTP request to the server.

[1124] Step 16:

[1125] Server: Receives the payment completion notification sent from the terminal. Based on the received information, sends a reservation completion and payment completion notification to the relevant store. Sends a completion notification to the terminal.

[1126] Step 17:

[1127] Terminal: Receives the reservation and payment completion notification sent from the server and displays it to the user.

[1128] Example 1

[1129] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1130] The present invention relates to a system that allows users to receive personalized beauty and fashion recommendations. In particular, the system analyzes a user's facial photograph and provides personalized recommendations, but the process is complicated, and making reservations and payments is often not easy. The challenge for such a system is to provide a user-friendly and efficient method for making reservations and payments.

[1131] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1132] In this invention, the server includes means for a user to upload a facial photo, means for transmitting the facial photo to the server, means for analyzing the facial photo using a generative AI model, means for generating suggestions tailored to the user's preferences based on the analysis results obtained by the generative AI model, means for transmitting the suggestions to a terminal, means for the user to make a reservation based on the suggestions, and means for making payment through a payment platform. This enables the user to receive personalized beauty and fashion suggestions using the generative AI model and to smoothly make reservations and payments.

[1133] "User" refers to an individual who uses the system to receive beauty and fashion suggestions.

[1134] "Means for uploading a facial photo" refers to a part of the system that provides a function for a user to take a new facial photo or select an existing facial photo using a smartphone or tablet and send it to the server.

[1135] "Means for sending facial photos to a server" refers to the function of transferring facial photo data uploaded from a terminal to a server using an HTTP request, etc.

[1136] A "generative AI model" refers to an algorithm or system that uses technologies such as deep learning to analyze facial photos and convert their features into numerical data.

[1137] "Means of generating suggestions that match the user's preferences based on the analysis results" refers to a function that combines facial photo feature data obtained using a generative AI model with preferences and trend information entered by the user to create optimal makeup, hairstyle, and fashion suggestions.

[1138] "Means for sending proposal results to a terminal" refers to a function for sending proposal content generated on a server to a user's terminal using a data format such as JSON.

[1139] The "means for making reservations" is a part of the system that includes a function that allows users to make online reservations for services at beauty salons, esthetic salons, fashion stores, etc. based on the suggestions.

[1140] "Means for making payments through a payment platform" means a part of the system that provides users with the ability to complete online payments for the services they select.

[1141] "Server" refers to a computer system that processes data received from a user's device and uses a generative AI model to analyze and make recommendations.

[1142] "Terminal" refers to a device used by a user to access the system, including smartphones and tablets.

[1143] An "HTTP request" refers to a communication protocol for sending data to a web server, and is used when sending data such as a photo of a person's face to a server.

[1144] "JSON format" is an abbreviation for JavaScript Object Notation, and refers to a format for structuring and describing data, and is used for sending proposal results, etc.

[1145] The present invention is a system that allows users to receive personalized beauty and fashion recommendations and smoothly make reservations and payments. Specific methods for implementing this system are described in detail below.

[1146] First, the user uses a device such as a smartphone or tablet. The user launches a dedicated application installed on the device and uploads a photo of their face through the application. Specifically, the user can take a new photo of their face using the camera function or select an existing photo from the gallery. The uploaded photo is temporarily stored on the device.

[1147] The device then sends the temporarily stored facial photo data to the server as an HTTP request. The server preprocesses the received facial photo data using an image processing algorithm and inputs this preprocessed data into the generative AI model. The generative AI model uses deep learning to convert the facial photo's features into numerical data and extract information such as contours, skin tone, and eye shape. The analysis results are stored in the server's internal database.

[1148] The server then retrieves the analysis results of the user's facial photo from its internal database and combines them with the preferences and current trends the user has entered through the application. Using this information, the generative AI model generates optimal makeup, hairstyle, and fashion recommendations. The generated recommendations are converted into JSON format and sent from the server to the device.

[1149] The user checks these suggestions on the device, selects their favorite hair salon, beauty salon, or fashion store, and makes a reservation. Specifically, the user enters the necessary information, such as the service content and desired date and time, into the reservation form within the application to confirm the reservation. The device sends the reservation information to the server, which then notifies the specified store of that information.

[1150] Finally, the user uses the payment platform to make an online payment. The terminal displays an interface for inputting payment information, and after the user inputs the required payment information, the payment is completed through the payment platform. The server receives a payment completion notification from the payment platform and sends the information to the terminal together with the reservation information.

[1151] For example, if a user is interested in natural makeup and a bob hairstyle, they can use the application to upload a photo of their face and input this information along with their desired style. The server uses a generative AI model to generate optimal makeup and hairstyle suggestions and presents them to the user. The user can then select a suggested salon, make a reservation, and complete payment online.

[1152] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1153] Program processing flow

[1154] Step 1:

[1155] Users launch the dedicated application on their smartphone, tablet, or other device, and can either take a new photo of their face using the camera function on the application's home screen, or select an existing photo from their gallery.

[1156] Input: User interaction via smartphone or tablet

[1157] Output: A face photo data to be uploaded

[1158] Step 2:

[1159] Users upload their facial photo data through the application interface, and by clicking the upload button, the facial photo data is temporarily saved on the device.

[1160] Input: Face photo data selected or taken by the user

[1161] Output: Temporarily saved face photo data

[1162] Step 3:

[1163] The device sends the temporarily stored facial photo data to the server in the form of an HTTP request. Specifically, it uses a REST API to send the data to the server in the form of multipart form data.

[1164] Input: Temporarily saved face photo data

[1165] Output: Face photo data sent to the server

[1166] Step 4:

[1167] The server inputs the received facial photo data into the generative AI model, checks the data format, and performs preprocessing if necessary before passing it to the AI ​​model.

[1168] Input: Face photo data received from the device

[1169] Output: Preprocessed facial photo data that is fed into the AI ​​model

[1170] Step 5:

[1171] The generative AI model analyzes facial photo data and extracts numerical data as features, such as facial contours, skin tone, and eye shape.

[1172] Input: Preprocessed facial photo data

[1173] Output: Extracted feature data

[1174] Step 6:

[1175] The server stores the feature data obtained from the generative AI model in an internal database, along with the beauty and fashion preferences and trend information entered by the user.

[1176] Input: Extracted feature data, user preferences and trend information

[1177] Output: Feature data and preference information stored in the internal database

[1178] Step 7:

[1179] The server retrieves feature data and preference information from an internal database, and then uses a generative AI model based on this to generate optimal makeup, hairstyle, and fashion suggestions.

[1180] Input: Feature data, user preferences and trend information

[1181] Output: Generated proposal data

[1182] Step 8:

[1183] The server converts the generated proposal data into JSON format and sends it to the device using a REST API, sending the encoded data packet to the device.

[1184] Input: Generated proposal data

[1185] Output: Proposal data in JSON format, data sent to the device

[1186] Step 9:

[1187] The device displays the received recommendation data to the user, who then checks the displayed recommendations and selects their preferred beauty salon, aesthetic salon, or fashion store.

[1188] Input: JSON formatted proposal data received from the server

[1189] Output: The suggestions displayed to the user

[1190] Step 10:

[1191] The user follows the suggestions, enters the necessary information (service details, date and time, etc.) into the reservation form within the application, and presses the reservation button to confirm the reservation.

[1192] Input: Booking information entered by the user

[1193] Output: Confirmed reservation information

[1194] Step 11:

[1195] The terminal sends the confirmed reservation information to the server, and the server notifies the store of the received reservation information.

[1196] Input: Reservation information confirmed by the user

[1197] Output: Reservation information sent to the server, reservation information notified to the target store

[1198] Step 12:

[1199] To make a payment using the payment platform, the user inputs the necessary payment information (such as card information) into the payment interface on the terminal, which then transmits this information to the payment platform for payment processing.

[1200] Input: User's payment information

[1201] Output: Payment information sent to the payment platform, payment processing results

[1202] Step 13:

[1203] The server receives a payment completion notification from the payment platform and transmits the information to the terminal together with the user's reservation information.

[1204] Input: Payment completion notification from payment platform

[1205] Output: Payment completion notification and reservation information sent to the user's terminal

[1206] Through these processing steps, users can receive personalized beauty and fashion recommendations using generative AI models, and make reservations and payments smoothly.

[1207] (Application example 1)

[1208] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1209] Systems that offer beauty and fashion recommendations are expected to enable users to quickly and accurately receive personalized recommendations, while also enabling smooth in-store reservations and payment. While conventional systems analyze users' facial photos to provide recommendations, these recommendations often do not adequately address individual preferences or trends. Furthermore, the reservation and payment processes are complicated, resulting in poor user convenience. Therefore, there is a need for a system that allows users to receive personalized recommendations and easily make reservations and payments.

[1210] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1211] In this invention, the server includes means for a user to upload a facial photo, means for transmitting the facial photo to the server, means for the server to analyze the facial photo using a generative AI model, means for suggesting makeup, hairstyles, and fashions that suit the user's preferences based on the analysis results obtained by the generative AI model, means for making a store reservation based on the suggested makeup, hairstyles, and fashions, means for making a payment through a payment platform, means for saving the user's data in an internal database, and interface means for checking the suggestion results on a smartphone. This allows the user to receive personalized beauty and fashion suggestions and smoothly make a reservation and make a payment at the suggested store.

[1212] "Means for users to upload facial photos" refers to the interface and functionality that allows users to select or take a photo of their face using a smartphone or other device and upload it into the application.

[1213] The "means for transmitting facial photographs to a server" refers to a communication protocol and interface for securely and efficiently transmitting uploaded facial photographs of users to a server.

[1214] "Means for the server to analyze facial photos using a generative AI model" refers to the function of inputting facial photo data received by the server into a generative AI model, which then analyzes facial features and extracts them as data.

[1215] "A means of suggesting makeup, hairstyles, and fashion that suit the user's preferences based on the analysis results obtained by a generative AI model" is a system that uses the results of facial photo analysis by a generative AI model to suggest optimal beauty and fashion options in combination with the user's preference information.

[1216] "Means for making store reservations based on suggested makeup, hairstyles, and fashion" refers to the interface and functionality for making reservations at hair salons, beauty salons, and fashion stores through the application based on the suggested results of the generative AI model.

[1217] "Means for making payments through a payment platform" means a payment system and interface for completing online payments for the booked services.

[1218] "Means for storing user data in an internal database" refers to a database system for securely storing user facial photographs and preference data for use in subsequent suggestions.

[1219] The "interface means for checking the proposal results on a smartphone" refers to a user interface that allows a user to visually check the beauty and fashion proposals from the generated AI model using a smartphone.

[1220] This invention is a system that allows users to receive personalized beauty and fashion suggestions, and smoothly make store reservations and payments. This system includes the following means.

[1221] A way for users to upload photos of their faces

[1222] The user starts the smartphone application, takes a photo of their face using the camera function or selects an existing photo from the gallery, and then uploads the photo data to the application through an interface for uploading a photo of their face.

[1223] A means of sending a face photo to the server

[1224] The device temporarily stores the uploaded facial photo data and prepares it for transmission to the server. The stored facial photo data is then sent to the server as an HTTP request. This is mainly done using the React Native HTTP request module.

[1225] The server generates a facial photo and analyzes it using an AI model.

[1226] The server inputs the facial photo data received from the device into the generative AI model. At this time, the server uses TensorFlow and PyTorch to analyze the facial photo's features (e.g., contours, skin tone, eye shape, etc.) and converts them into numerical data. The analysis results are then stored in an internal database (MongoDB) and transferred for subsequent processing.

[1227] How generative AI models generate proposals

[1228] The server retrieves the results of the user's facial photo analysis from an internal database, inputs the user's preferences and current trend information into the AI ​​model, and the generative AI model generates optimal suggestions for makeup, hairstyles, and fashion, and sends the generated suggestions in JSON format to the device.

[1229] A way to make restaurant reservations based on the proposed content

[1230] The user checks the suggestions and selects the hair salon, aesthetic salon, or fashion store that they like. They then enter reservation information, such as the service content and date and time of the selected store, and confirm the reservation.

[1231] A means of making payments through a payment platform

[1232] The terminal sends the reservation information entered by the user to the server. It then displays an interface for entering payment information, and the user enters the payment information. The entered payment information is sent to a payment platform (e.g., Stripe API) and the processing result is received. The server notifies the store of the reservation information received from the terminal, receives a payment completion notification from the payment platform, and confirms the completion of the reservation. The completed reservation and payment information are sent to the terminal.

[1233] Specific example explanation

[1234] For example, a user launches the "Beauty360" app on their smartphone and uploads a photo of their face. They then enter "I'm interested in natural makeup and bob hairstyles" through the interface. The server analyzes the photo and uses this information to generate optimal recommendations using a generative AI model. The user then confirms the recommendations, makes a reservation at the suggested salon, and makes payment via Stripe.

[1235] Example prompt sentence:

[1236] {

[1237] "face_features": {

[1238] "face_shape": "round",

[1239] "skin_tone": "fair",

[1240] "eye_shape": "almond"

[1241] },

[1242] "user_preference": {

[1243] "style": "natural",

[1244] "interest": ["hair", "makeup"]

[1245] },

[1246] "current_trends": {

[1247] "makeup": ["minimalist", "soft colors"],

[1248] "hair": ["long bob", "wavy"]

[1249] }

[1250] }

[1251] By writing and implementing it in this way, users will be able to receive personalized beauty and fashion suggestions, and easily make reservations and payments at physical stores using a smartphone application.

[1252] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1253] Step 1:

[1254] A user starts the smartphone application and takes a face photo using the camera function or selects an existing face photo from the gallery. This inputs face photo data. Next, the face photo data is uploaded to the application through the face photo upload interface and temporarily saved on the device.

[1255] Step 2:

[1256] The device sends the temporarily saved facial photo data to the server as an HTTP request. The server receives the facial photo data and prepares to start processing it as input data.

[1257] Step 3:

[1258] After receiving the facial photo data, the server uses TensorFlow to input the facial photo into a generative AI model. The generative AI model analyzes the facial photo's features (contours, skin tone, eye shape, etc.) and converts these features into numerical data. The converted data is stored in an internal database (e.g., MongoDB) and transferred for subsequent processing.

[1259] Step 4:

[1260] The server retrieves the results of the user's facial photo analysis from its internal database and inputs them into the generative AI model along with the user's preferences and trend information. The generative AI model then generates makeup, hairstyle, and fashion suggestions and outputs them in JSON format. The output suggestions are then sent to the device.

[1261] Step 5:

[1262] The user checks the recommendations through their smartphone application. At this time, the user interface visually displays the beauty recommendations created by the generative AI model. The user then selects their favorite beauty salon, beauty salon, or fashion store.

[1263] Step 6:

[1264] The user inputs the service details and date and time of the selected store and sends the reservation information to the server. The server notifies the corresponding store of the received reservation information. The notified store confirms the reservation and sends the reservation confirmation information to the server.

[1265] Step 7:

[1266] The terminal sends the reservation information entered by the user to the server, and then displays an interface for entering payment information. The user enters the payment information, and the terminal sends the information to a payment platform (e.g., Stripe API) for processing. The processing result is sent to the server.

[1267] Step 8:

[1268] The server receives a payment completion notification from the payment platform and confirms the completion of the reservation. This completion information is sent to the terminal so that the user can check it. This completes the entire process.

[1269] By following the above steps, users can receive personalized beauty and fashion recommendations and smoothly make reservations and payments. Examples of prompts are as follows:

[1270] {

[1271] "face_features": {

[1272] "face_shape": "round",

[1273] "skin_tone": "fair",

[1274] "eye_shape": "almond"

[1275] },

[1276] "user_preference": {

[1277] "style": "natural",

[1278] "interest": ["hair", "makeup"]

[1279] },

[1280] "current_trends": {

[1281] "makeup": ["minimalist", "soft colors"],

[1282] "hair": ["long bob", "wavy"]

[1283] }

[1284] }

[1285] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1286] The present invention is a system that provides beauty and fashion recommendations based on a user's facial photo and preferences, and allows for smooth reservations and payments. It also has the ability to recognize the user's emotional state and make optimal recommendations based on that. The following elements are required to implement this system:

[1287] A way for users to upload photos of their faces

[1288] User:

[1289] 1. Launch the application and take a photo of your face using the camera function or select an existing photo from your gallery.

[1290] 2. Upload your face photo data to the application through the interface for uploading face photos.

[1291] A means of sending a face photo to the server

[1292] Device:

[1293] 1. The uploaded facial photo data is temporarily saved and prepared for sending to the server.

[1294] 2. The saved facial photo data is sent to the server as an HTTP request.

[1295] The server generates a facial photo and analyzes it using an AI model.

[1296] server:

[1297] 1. The facial photo data received from the device is input into the generative AI model.

[1298] 2. The generative AI model analyzes the facial features (e.g., facial contours, skin tone, eye shape, etc.) and converts them into numerical data.

[1299] 3. The analysis results are stored in an internal database for subsequent processing.

[1300] Using an emotion engine to recognize user emotions

[1301] server:

[1302] 1. Input facial photo data into the emotion engine to analyze the user's emotional state (e.g., happiness, sadness, surprise, etc.).

[1303] 2. The analysis results of the emotion engine are stored in an internal database.

[1304] How generative AI models generate proposals

[1305] server:

[1306] 1. Obtain the analysis results and emotional state of the user's face photo from the internal database.

[1307] 2. User-input preferences and current trend information are fed into the AI ​​model to generate optimal makeup, hairstyle, and fashion suggestions.

[1308] 3. It also dynamically adjusts its suggestions based on your emotional state.

[1309] 4. The generated proposal results are sent to the device.

[1310] A way to make a reservation based on the suggestions

[1311] User:

[1312] 1. Check the suggestions and select your favorite hair salon, beauty salon, or fashion store.

[1313] 2. Enter reservation information such as the service content, date and time of the selected store and confirm your reservation.

[1314] A means of making payments through a payment platform

[1315] Device:

[1316] 1. The reservation information entered by the user is sent to the server.

[1317] 2. An interface for entering payment information is displayed, and the user enters the payment information.

[1318] 3. The entered payment information is sent to the payment platform and the processing result is received.

[1319] server:

[1320] 1. Notify beauty salons, esthetic salons, and fashion stores of reservation information received from the device.

[1321] 2. Receive a payment confirmation from the payment platform to confirm the completion of your booking.

[1322] 3. Send the completed booking and payment information to the terminal.

[1323] Specific examples

[1324] Example 1: Makeup and hairstyle suggestions

[1325] User:

[1326] 1. Upload a photo of your face using the application and indicate that you are interested in natural makeup and bob hairstyles.

[1327] 2. Check the suggestions from the server and make a reservation at the suggested beauty salon.

[1328] Device:

[1329] 1. Send a face photo and user preferences to the server.

[1330] 2. Receive suggestions from the server and present them to the user.

[1331] 3. Send the reservation information to the server and display the payment interface.

[1332] server:

[1333] 1. Analyze your facial photo and suggest the best makeup and hairstyle.

[1334] 2. The emotion engine analyzes the user's emotional state and adjusts the suggestions accordingly.

[1335] 3. Notify the beauty salon of the user's reservation information and receive payment completion information.

[1336] Example 2: Fashion coordination suggestions

[1337] User:

[1338] 1. Enter your interest in casual style and upload a photo of your face.

[1339] 2. Make an appointment at the suggested fashion store and complete the payment online.

[1340] Device:

[1341] 1. Send your face photo and preference information to the server.

[1342] 2. Receive suggestions from the server and present them to the user.

[1343] 3. Send the reservation information to the server and process the payment.

[1344] server:

[1345] 1. We suggest the best fashion coordination based on the analysis of your face photo and your preferences.

[1346] 2. The emotion engine analyzes the user's emotional state and adjusts the suggestions accordingly.

[1347] 3. The user's reservation information is notified to the fashion store and payment completion information is received.

[1348] As described above, the system of the present invention allows users to receive personalized beauty and fashion suggestions using generative AI models and emotion engines, and to easily make reservations and payments.

[1349] The processing flow will be explained below.

[1350] Step 1:

[1351] User: Launches the application and takes a photo of his / her face using the camera function or selects an existing photo from the gallery. Clicks the button to upload a photo of his / her face.

[1352] Step 2:

[1353] Terminal: The facial photo data selected or taken by the user is temporarily stored within the application. An HTTP request is generated to send the stored facial photo data to the server. The generated HTTP request is sent to the server.

[1354] Step 3:

[1355] Server: Receives facial photo data sent from the device. Preprocesses the facial photo data to pass it to the generative AI model and emotion engine.

[1356] Step 4:

[1357] Server: The generative AI model analyzes the facial features (e.g., facial contours, skin tone, eye shape, etc.) and converts them into numerical data. The analysis results are stored in an internal database.

[1358] Step 5:

[1359] Server: Inputs a facial photo into the emotion engine and analyzes the user's emotional state (e.g., happiness, sadness, surprise, etc.). The analysis results of the emotion engine are also stored in an internal database.

[1360] Step 6:

[1361] User: Fills out a form to select their makeup, hairstyle and fashion preferences; answers choices and questions about current trends; completes the form and clicks the "Submit" button.

[1362] Step 7:

[1363] Terminal: The application temporarily stores the preferences and trend information entered by the user. It generates an HTTP request to send the stored data to the server. It then sends the generated HTTP request to the server.

[1364] Step 8:

[1365] Server: Receives preferences and trend information sent by the user. Combining the results of facial photo analysis, emotional state, and preference information, the generative AI model generates optimal makeup, hairstyle, and fashion suggestions. The suggestions are dynamically adjusted based on the user's emotional state.

[1366] Step 9:

[1367] Server: Sends the proposed results from the generative AI model to the terminal.

[1368] Step 10:

[1369] Terminal: Receives the proposal results sent from the server. The received proposal results are displayed within the application and confirmed by the user.

[1370] Step 11:

[1371] User: Check the suggestions and select a hair salon, beauty salon, or fashion store that suits them best. Enter the reservation information (date, time, service details, etc.) for the selected store and click the "Confirm reservation" button.

[1372] Step 12:

[1373] Terminal: The reservation information entered by the user is temporarily saved within the application. An HTTP request is generated to send the reservation information to the server. The generated HTTP request is sent to the server.

[1374] Step 13:

[1375] Server: Receives reservation information sent from the terminal. Generates and sends another HTTP request to the store's system to notify the corresponding store of the received reservation information. Receives reservation confirmation or adjustment information from the store and stores it.

[1376] Step 14:

[1377] Server: Sends reservation confirmation information to the terminal.

[1378] Step 15:

[1379] Terminal: Receives the reservation confirmation information sent from the server and displays it to the user. Generates a link to the payment platform and presents it to the user.

[1380] Step 16:

[1381] User: Click on the payment platform link displayed on the terminal, select the payment method, enter the required payment information, and click the "Complete Payment" button.

[1382] Step 17:

[1383] Terminal: Sends the payment information entered by the user to the payment platform. Receives the payment completion notification and generates an HTTP request to send to the server. Sends the generated HTTP request to the server.

[1384] Step 18:

[1385] Server: Receives the payment completion notification sent from the terminal. Based on the received information, sends a reservation completion and payment completion notification to the relevant store. Sends a completion notification to the terminal.

[1386] Step 19:

[1387] Terminal: Receives the reservation and payment completion notification sent from the server and displays it to the user.

[1388] Example 2

[1389] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1390] Conventional beauty and fashion recommendation systems have difficulty providing personalized recommendations that comprehensively consider a user's facial photo, emotional state, preferences, etc. Furthermore, there has been a lack of systems that allow users to instantly complete reservations and payments based on the recommendations. Therefore, there has been a strong demand for the development of a system that allows users to easily receive optimal beauty and fashion recommendations and quickly use the service.

[1391] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1392] In this invention, the server includes means for using an emotion engine that analyzes the user's emotional state, means for adjusting the content of suggestions according to the emotional state, and means for suggesting makeup, hairstyles, and fashions that suit the user's preferences based on the analysis results obtained by the generative AI model. This makes it possible to quickly provide personalized and optimal suggestions by comprehensively considering the user's facial photo and emotional state, and to smoothly execute the reservation and payment processes.

[1393] A "user" is a person who uses the system to upload a photo of their face and receive beauty and fashion suggestions.

[1394] A "face photo" is still image data that includes features such as the user's facial contours, eye shape, and skin tone.

[1395] The "server" is a central processing unit that processes facial photo data, user preferences, and emotional state, and generates optimal suggestions using generative AI models and emotion engines.

[1396] A "generative AI model" is an artificial intelligence model that analyzes facial photos and converts their features into numerical data.

[1397] An "emotion engine" is software that analyzes a user's emotional state (e.g., happiness, sadness, surprise, etc.) from a photograph of their face.

[1398] "Makeup" refers to methods and products for applying makeup to a user's face.

[1399] A "hairstyle" is a method or design for instructing the shape and arrangement of a user's hair.

[1400] "Fashion" refers to a user's clothing, accessories, and other decorative items related to their appearance.

[1401] A "reservation" is a procedure in which a user determines in advance the date, time, and content of services to be used at a beauty salon, esthetic salon, or fashion store.

[1402] "Payment Platform" means an online payment system through which Users pay for Services.

[1403] "Suggestions" are specific advice and options regarding makeup, hairstyles, and fashion provided to users based on the results of analysis by generative AI models and emotion engines.

[1404] MODE FOR CARRYING OUT THE INVENTION

[1405] This system provides beauty and fashion recommendations based on a user's facial photograph and preferences, and allows for smooth booking and payment. It also has the ability to recognize the user's emotional state and provide optimal recommendations based on that. The specific configuration and operating procedures for implementing this system are described below.

[1406] A way for users to upload photos of their faces

[1407] The user launches the application on their smartphone or PC and takes a photo of their face using the camera function, or selects an existing photo from the gallery. Then, the user presses the upload button on the application to upload the photo data to the application. This application can run on the iOS or Android platform, for example.

[1408] A means of sending a face photo to the server

[1409] The device temporarily stores the uploaded facial photo data and sends it to the server in the form of an HTTP request. The HTTP request header contains authentication information, and the body contains the facial photo data. The device's local storage is used for temporary storage.

[1410] The server generates a facial photo and analyzes it using an AI model.

[1411] The server inputs the facial photo data received from the device into a generative AI model. The server performs preprocessing on the facial photo data, such as resizing and color normalization. The data is then input into a specialized facial analysis AI model (e.g., a convolutional neural network using TensorFlow) to convert features such as facial contours, skin tone, and eye shape into numerical data. The results of this analysis are stored in an internal database and used for subsequent processing.

[1412] Using an emotion engine to recognize user emotions

[1413] The server inputs the facial photo data into an emotion engine to analyze the user's emotional state (e.g., happiness, sadness, surprise, etc.). The emotion engine's analysis results are also stored in an internal database. The emotion engine uses, for example, facial expression recognition software (e.g., Microsoft's Azure Emotion API).

[1414] How generative AI models generate proposals

[1415] The server retrieves the analysis results of the user's facial photo and emotional state from an internal database, inputs the user's preferences and current trend information into the AI ​​model, and generates optimal suggestions for makeup, hairstyles, and fashion. Furthermore, the suggestions are dynamically adjusted according to the user's emotional state. The results of these suggestions are returned to the device in JSON format.

[1416] For example, if a user enters "I'm interested in natural makeup and bob hairstyles" and uploads a photo of their face, the server will use a generative AI model to analyze the user's facial features and make optimal suggestions for natural makeup and bob hairstyles based on those features.

[1417] Example prompt sentence:

[1418] "I'm interested in natural makeup and bob hairstyles."

[1419] A way to make a reservation based on the suggestions

[1420] The user reviews the suggestions and selects their favorite beauty salon, esthetic salon, or fashion store on the application. They then enter reservation information such as the service content and date and time of the selected store and press "Confirm reservation" to make the reservation. The reservation information is then sent from the device to the server.

[1421] A means of making payments through a payment platform

[1422] The terminal sends the reservation information entered by the user to the server, and also displays an interface for entering payment information, where the user enters payment information such as credit card information. The terminal sends data to the API of the payment platform (e.g., Stripe or PayPal) and receives the processing results.

[1423] A means by which the server processes booking and payment information

[1424] The server notifies the beauty salon, esthetic salon, or fashion store of the reservation information received from the device. It also receives a payment completion notification from the payment platform and confirms the completion of the reservation. The completed reservation and payment information are returned to the device in JSON format.

[1425] As described above, the system of the present invention allows users to receive personalized beauty and fashion suggestions using generative AI models and emotion engines, and to easily make reservations and payments.

[1426] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1427] Step 1:

[1428] Users launch the application on their smartphone or PC, take a photo of their face using the camera function, or select an existing photo from the gallery, and then press the upload button to upload the photo data to the application.

[1429] Input: Facial photo data taken or selected by the user

[1430] Output: Facial photo data sent to the application through the upload interface

[1431] Specific actions: The user taps the app's camera icon to launch the camera, takes a photo of their face, and taps "Done." Alternatively, the user taps the gallery icon, selects an existing photo, and presses the "Upload" button.

[1432] Step 2:

[1433] The device temporarily stores the uploaded facial photo data and sends it to the server in the form of an HTTP request.

[1434] Input: Uploaded face photo data

[1435] Output: Facial photo data sent to the server in the form of an HTTP request

[1436] Specific operation: The device temporarily stores the face photo data in local storage and then sends it to the server's API endpoint in the background. The HTTP request header contains authentication information, and the body contains the face photo data.

[1437] Step 3:

[1438] The server inputs the facial photo data received from the device into the generative AI model. The server preprocesses the facial photo data by resizing and color normalizing it, and then inputs the data into the generative AI model.

[1439] Input: Facial photo data received from the device

[1440] Output: Analysis results converted from facial features into numerical data

[1441] Specific operation: The server performs preprocessing on the facial photo data, such as resizing and color normalizing, and then inputs the data into a generative AI model (e.g., a convolutional neural network using TensorFlow), converting features such as facial contours, skin tone, and eye shape into numerical data.

[1442] Step 4:

[1443] The server inputs the facial photo data into an emotion engine to analyze the user's emotional state (e.g., happiness, sadness, surprise, etc.).

[1444] Input: Facial photo data

[1445] Output: Analysis results showing the user's emotional state

[1446] Specific operation: The server inputs facial photo data into an emotion engine (e.g., facial expression recognition software), analyzes the emotional state, and stores the results in an internal database.

[1447] Step 5:

[1448] The server retrieves the analysis results of the user's facial photo and emotional state from an internal database, and inputs the user's input preferences and current trend information into an AI model to generate optimal suggestions for makeup, hairstyles, and fashion.

[1449] Input: User's facial photo analysis results, emotional state, preferences and trend information

[1450] Output: Makeup, hairstyle, and fashion suggestions

[1451] Specific operation: The server retrieves the necessary data from the internal database, integrates them, and inputs them into the generative AI model. The generative AI model generates a proposal, adjusts the proposal content using the emotion engine, and sends it to the device in JSON format.

[1452] Step 6:

[1453] The user checks the suggestions on the application and selects the hair salon, beauty salon, or fashion store that suits them best. After that, they enter reservation information such as the service content, date and time, and confirm the reservation.

[1454] Input: Proposal details, reservation information (service details, date and time)

[1455] Output: Confirmed reservation information

[1456] Specific operation: The user checks the list of suggestions, taps the selection button to move to the reservation page, enters the service details and date and time, and taps "Confirm reservation."

[1457] Step 7:

[1458] The terminal transmits the reservation information entered by the user to the server, displays an interface for entering payment information, and the user enters the payment information. The terminal then transmits the data to the payment platform and receives the processing result.

[1459] Input: Reservation information, payment information

[1460] Output: Processing result by payment platform

[1461] Specific operation: The terminal sends the reservation information to the server, then displays the payment page. The user enters credit card information and taps the "Pay" button. The terminal then sends the data to the payment platform's API and receives the processing result.

[1462] Step 8:

[1463] The server notifies the beauty salon, esthetic salon, or fashion store of the reservation information received from the terminal, receives a payment completion notification from the payment platform, confirms the completion of the reservation, and sends the completed reservation and payment information to the terminal.

[1464] Input: Reservation information, payment completion notification

[1465] Output: Reservation completion information, payment completion information

[1466] Specific operation: The server notifies the store management system of the reservation information in real time, then receives a payment completion notification from the payment platform, updates the reservation status to "Completed", and returns the reservation and payment completion information to the terminal in JSON format.

[1467] (Application example 2)

[1468] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1469] Conventional beauty and fashion recommendation systems were unable to provide personalized recommendations that fully considered the individual characteristics and emotional state of each user. As a result, users were often dissatisfied with the recommendations and stopped using the service. In addition, the reservation and payment procedures were cumbersome, leaving a need for an improved user experience.

[1470] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1471] In this invention, the server includes means for a user to upload a facial photo, means for transmitting the facial photo to the server, means for the server to use a generative AI model that analyzes the facial photo, means for suggesting beauty and fashion that matches the user's preferences based on the analysis results obtained by the generative AI model, means for making a reservation at a service facility based on the suggested beauty and fashion, means for making a payment through a payment platform, means for inputting the user's facial photo and preference information through a smartphone application, means for recognizing the user's emotional state using an emotion engine, and means for dynamically adjusting the content of the proposals according to the emotional state. This not only enables personalized proposals based on the user's individual characteristics and emotional state, but also enables a smooth process from reservation to payment.

[1472] "Means for users to upload facial photos" refers to the methods or functions that allow users to take or select their own facial photos and transmit the photo data via the Internet.

[1473] "Means for sending facial photos to a server" refers to a system that has the function of temporarily storing facial photos taken or selected by a user and sending them to an external server via the Internet.

[1474] "Means of using generative AI models" refers to methods or functions that use models to generate results by using artificial intelligence to analyze collected data.

[1475] "Means of suggesting beauty and fashion that suits the user's preferences" refers to a function that suggests beauty, hairstyles, and fashion that suits the user's individual preferences based on the analysis results obtained by the generative AI model.

[1476] "Means for making reservations at service providing facilities" refers to a function that enables users to make online reservations at service providing facilities such as beauty salons, esthetic salons, and fashion stores based on the beauty and fashion recommendations.

[1477] "Means of making payments through a payment platform" refers to the use of payment systems and services to make payments for online reservations in a secure and efficient manner.

[1478] "Means for inputting a user's facial photograph and preferred information via a smartphone application" refers to an interface that allows a user to input a photograph of their face and preferred beauty and fashion information via an application that runs on a smartphone.

[1479] "Means for recognizing a user's emotional state using an emotion engine" refers to a method that uses a system or algorithm to analyze a facial photograph and determine the user's current emotional state (such as joy, sadness, surprise, etc.).

[1480] "Means for dynamically adjusting the content of suggestions according to the emotional state" refers to a function that appropriately changes and adjusts the content of beauty and fashion suggestions in real time based on the recognized emotional state of the user.

[1481] The present invention is a system in which a user uploads a photo of their face via a smartphone application, and the server analyzes the photo using a generated AI model to suggest beauty and fashion products that match the user's preferences. This system is realized through the following configuration and processing.

[1482] The server first receives a facial photo taken or selected by the user using a smartphone application. The facial photo is uploaded using the smartphone's camera or gallery function. This photo data is then sent to the server via the Internet.

[1483] The server temporarily stores the received facial photo data and analyzes it using a generative AI model. The analysis extracts facial features such as facial contours, skin tone, and eye shape as numerical data. This process uses machine learning frameworks such as TensorFlow.

[1484] Furthermore, the server uses an emotion engine to recognize the user's emotional state. The emotion engine can determine the user's current emotion (e.g., joy, sadness, surprise, etc.) from the facial photograph. This information is also stored in an internal database as numerical data.

[1485] The server uses a generative AI model to generate optimal beauty and fashion recommendations based on the user's preferences, current trends, and the acquired facial features and emotional state. The recommendations are dynamically adjusted according to the user's emotional state and presented in a form optimized for each individual user. The recommendations are displayed to the user via a smartphone application.

[1486] Users can check the suggestions and make reservations at beauty salons, esthetic salons, and fashion stores that they are interested in. They enter reservation information through the smartphone application and confirm the reservation. The reservation information is sent to the server, which then notifies the relevant service provider.

[1487] Finally, payment is processed. The smartphone application connects to a payment platform and securely processes the payment information entered by the user. Services such as Stripe can be used as payment platforms. A notification of payment completion is sent to the server, confirming the completion of the reservation.

[1488] As a concrete example, consider the following prompt sentence:

[1489] Example prompt sentence:

[1490] "User's face photo has been uploaded and preprocessed. Generate a personalized beauty or fashion suggestion based on the following parameters:

[1491] Face features: [numerical data such as face shape, skin tone, eye shape, etc.]

[1492] Emotion state: "Joy"

[1493] User preferences: ["Natural makeup", "Bob hairstyle"]

[1494] Current trends: ["Spring 2023 trends"]

[1495] This will enable users to receive beauty and fashion suggestions based on their facial photos, preferences, and real-time emotional state, and will also enable them to complete reservations and payments on the spot.

[1496] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1497] Step 1:

[1498] The user starts the smartphone application and takes a photo of their face or selects one from the gallery. They then use the application's camera or gallery function to obtain a photo of their face and press the upload button to enter the photo data. The entered photo data is temporarily saved.

[1499] Step 2:

[1500] The device sends face photo data to the server. The face photo data is temporarily saved and then sent to the server via an HTTP request. The input in this step is the face photo data, and the output is the data sent to the server.

[1501] Step 3:

[1502] The server inputs the received facial photo data into a generative AI model for analysis. Using a machine learning framework such as TensorFlow, facial photo features (contours, skin tone, eye shape, etc.) are extracted as numerical data. The input is facial photo data, and the output is numerical data indicating facial features.

[1503] Step 4:

[1504] The server uses an emotion engine to recognize the user's emotional state. The received facial photo data is input into the emotion engine to determine the user's emotional state (e.g., joy, sadness, surprise, etc.). The emotion engine's specific operation uses an algorithm to analyze facial expressions and micro-expressions. The input is facial photo data, and the output is data indicating the user's emotional state.

[1505] Step 5:

[1506] The server generates beauty and fashion suggestions using a generative AI model based on facial feature data, emotional state data, user preference information, and trend information. This uses prompts that take into account the user's input preferences (e.g., natural makeup, bob hairstyle) and current trend information. Specific examples of generated prompts include:

[1507] "User's face photo has been uploaded and preprocessed. Generate a personalized beauty or fashion suggestion based on the following parameters:

[1508] Face features: [numerical data such as face shape, skin tone, eye shape, etc.]

[1509] Emotion state: "Joy"

[1510] User preferences: ["Natural makeup", "Bob hairstyle"]

[1511] Current trends: ["Spring 2023 trends"]

[1512] The inputs are facial feature data, emotional state data, user preference information, and trend information, and the output is suggested beauty and fashion content.

[1513] Step 6:

[1514] The user checks the proposed beauty and fashion details on a smartphone application and selects the service provider of interest (beauty salon, esthetic salon, fashion store). The user then enters the reservation information for the desired service (date, time, location, service details, etc.) and presses the reservation button. The input is the confirmation result of the proposal details and the reservation information, and the output is the entered reservation information.

[1515] Step 7:

[1516] The terminal sends the entered reservation information to the server, which then notifies the relevant service provider of that information. The input is the reservation information, and the output is a notification to the service provider.

[1517] Step 8:

[1518] A user enters payment information into a smartphone application to make a payment. The application then sends the entered payment information to a payment platform (e.g., Stripe) to process the payment. The input is the payment information, and the output is the payment processing result.

[1519] Step 9:

[1520] The server receives the payment completion notification and confirms the completion of the reservation. It then notifies the user of the completed reservation information and payment information. The input is the payment processing result, and the output is the completed reservation information and payment completion notification.

[1521] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1522] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1523] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[1524] [Fourth embodiment]

[1525] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[1526] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1527] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1528] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[1529] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1530] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1531] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1532] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[1533] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1534] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1535] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1536] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1537] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1538] The present invention is a system that allows users to receive personalized beauty and fashion recommendations and smoothly make reservations and payments. The following elements are required to implement this system:

[1539] A way for users to upload photos of their faces

[1540] User:

[1541] 1. Launch the application and take a photo of your face using the camera function or select an existing photo from your gallery.

[1542] 2. Upload the facial photo data to the application through the interface for uploading facial photos.

[1543] A means of sending a face photo to the server

[1544] Device:

[1545] 1. The uploaded facial photo data is temporarily saved and prepared for sending to the server.

[1546] 2. The saved facial photo data is sent to the server as an HTTP request.

[1547] The server generates a facial photo and analyzes it using an AI model.

[1548] server:

[1549] 1. The facial photo data received from the device is input into the generative AI model.

[1550] 2. The generative AI model analyzes the facial features (e.g., facial contours, skin tone, eye shape, etc.) and converts them into numerical data.

[1551] 3. Store the analysis results in an internal database and forward them for further processing.

[1552] How generative AI models generate proposals

[1553] server:

[1554] 1. Obtain the analysis results of the user's facial photo from the internal database.

[1555] 2. User-input preferences and current trend information are fed into the AI ​​model to generate optimal makeup, hairstyle, and fashion suggestions.

[1556] 3. The generated proposal results are sent to the device in JSON format.

[1557] A way to make a reservation based on the suggestions

[1558] User:

[1559] 1. Check the suggestions and select your favorite hair salon, beauty salon, or fashion store.

[1560] 2. Enter reservation information such as the service content, date and time of the selected store and confirm your reservation.

[1561] A means of making payments through a payment platform

[1562] Device:

[1563] 1. The reservation information entered by the user is sent to the server.

[1564] 2. An interface for entering payment information is displayed, and the user enters the payment information.

[1565] 3. The entered payment information is sent to the payment platform and the processing result is received.

[1566] server:

[1567] 1. Notify beauty salons, esthetic salons, and fashion stores of reservation information received from the device.

[1568] 2. Receive a payment confirmation from the payment platform to confirm the completion of your booking.

[1569] 3. Send the completed booking and payment information to the terminal.

[1570] Specific examples

[1571] Example 1: Makeup and hairstyle suggestions

[1572] User:

[1573] 1. Upload a photo of your face using the application and indicate that you are interested in natural makeup and bob hairstyles.

[1574] 2. Check the suggestions from the server and make a reservation at the suggested beauty salon.

[1575] Device:

[1576] 1. Send a face photo and user preferences to the server.

[1577] 2. Receive suggestions from the server and present them to the user.

[1578] 3. Send the reservation information to the server and display the payment interface.

[1579] server:

[1580] 1. Analyze your facial photo and suggest the best makeup and hairstyle.

[1581] 2. Notify the beauty salon of the user's reservation information and receive payment completion information.

[1582] Example 2: Fashion coordination suggestions

[1583] User:

[1584] 1. Enter your interest in casual style and upload a photo of your face.

[1585] 2. Make an appointment at the suggested fashion store and complete the payment online.

[1586] Device:

[1587] 1. Send your face photo and preference information to the server.

[1588] 2. Receive suggestions from the server and present them to the user.

[1589] 3. Send the reservation information to the server and process the payment.

[1590] server:

[1591] 1. We suggest the best fashion coordination based on the analysis of your face photo and your preferences.

[1592] 2. The user's reservation information is notified to the fashion store and payment completion information is received.

[1593] As described above, the system of the present invention allows users to receive personalized beauty and fashion suggestions using generative AI models, and easily make reservations and payments.

[1594] The processing flow will be explained below.

[1595] Step 1:

[1596] User: Launches the application and takes a photo of his / her face using the camera function or selects an existing photo. Clicks the button to upload a photo of his / her face.

[1597] Step 2:

[1598] Terminal: The facial photo data selected or taken by the user is temporarily stored within the application. An HTTP request is generated to send the stored facial photo data to the server. The generated HTTP request is sent to the server.

[1599] Step 3:

[1600] Server: Receives facial photo data sent from the device. The facial photo data is input into a generative AI model to analyze the user's facial features, contours, skin tone, etc. The analysis results are temporarily stored and used for subsequent processing.

[1601] Step 4:

[1602] User: Fills out a form to select their makeup, hairstyle and fashion preferences; answers choices and questions about current trends; completes the form and clicks the "Submit" button.

[1603] Step 5:

[1604] Terminal: The application temporarily stores the preferences and trend information entered by the user. It generates an HTTP request to send the stored data to the server. It then sends the generated HTTP request to the server.

[1605] Step 6:

[1606] Server: Receives preferences and trend information sent by the user. Combining the facial photo analysis results with preference information, the generative AI model generates optimal makeup, hairstyle, and fashion suggestions. The generated suggestions are temporarily stored.

[1607] Step 7:

[1608] Server: Sends the proposed results from the generative AI model to the terminal.

[1609] Step 8:

[1610] Terminal: Receives the proposal results sent from the server. The received proposal results are displayed within the application and confirmed by the user.

[1611] Step 9:

[1612] User: Check the suggestions and select a hair salon, beauty salon, or fashion store that suits them best. Enter the reservation information (date, time, service details, etc.) for the selected store and click the "Confirm reservation" button.

[1613] Step 10:

[1614] Terminal: The reservation information entered by the user is temporarily saved within the application. An HTTP request is generated to send the saved reservation information to the server. The generated HTTP request is sent to the server.

[1615] Step 11:

[1616] Server: Receives reservation information sent from the terminal. Generates and sends another HTTP request to the store's system to notify the corresponding store of the received reservation information. Receives reservation confirmation or adjustment information from the store and stores it.

[1617] Step 12:

[1618] Server: Sends reservation confirmation information to the terminal.

[1619] Step 13:

[1620] Terminal: Receives the reservation confirmation information sent from the server and displays it to the user. Generates a link to the payment platform and presents it to the user.

[1621] Step 14:

[1622] User: Click on the payment platform link displayed on the terminal, select the payment method, enter the required payment information, and click the "Complete Payment" button.

[1623] Step 15:

[1624] Terminal: Sends the payment information entered by the user to the payment platform. Receives the payment completion notification and generates an HTTP request to send to the server. Sends the generated HTTP request to the server.

[1625] Step 16:

[1626] Server: Receives the payment completion notification sent from the terminal. Based on the received information, sends a reservation completion and payment completion notification to the relevant store. Sends a completion notification to the terminal.

[1627] Step 17:

[1628] Terminal: Receives the reservation and payment completion notification sent from the server and displays it to the user.

[1629] Example 1

[1630] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1631] The present invention relates to a system that allows users to receive personalized beauty and fashion recommendations. In particular, the system analyzes a user's facial photograph and provides personalized recommendations, but the process is complicated, and making reservations and payments is often not easy. The challenge for such a system is to provide a user-friendly and efficient method for making reservations and payments.

[1632] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1633] In this invention, the server includes means for a user to upload a facial photo, means for transmitting the facial photo to the server, means for analyzing the facial photo using a generative AI model, means for generating suggestions tailored to the user's preferences based on the analysis results obtained by the generative AI model, means for transmitting the suggestions to a terminal, means for the user to make a reservation based on the suggestions, and means for making payment through a payment platform. This enables the user to receive personalized beauty and fashion suggestions using the generative AI model and to smoothly make reservations and payments.

[1634] "User" refers to an individual who uses the system to receive beauty and fashion suggestions.

[1635] "Means for uploading a facial photo" refers to a part of the system that provides a function for a user to take a new facial photo or select an existing facial photo using a smartphone or tablet and send it to the server.

[1636] "Means for sending facial photos to a server" refers to the function of transferring facial photo data uploaded from a terminal to a server using an HTTP request, etc.

[1637] A "generative AI model" refers to an algorithm or system that uses technologies such as deep learning to analyze facial photos and convert their features into numerical data.

[1638] "Means of generating suggestions that match the user's preferences based on the analysis results" refers to a function that combines facial photo feature data obtained using a generative AI model with preferences and trend information entered by the user to create optimal makeup, hairstyle, and fashion suggestions.

[1639] "Means for sending proposal results to a terminal" refers to a function for sending proposal content generated on a server to a user's terminal using a data format such as JSON.

[1640] The "means for making reservations" is a part of the system that includes a function that allows users to make online reservations for services at beauty salons, esthetic salons, fashion stores, etc. based on the suggestions.

[1641] "Means for making payments through a payment platform" means a part of the system that provides users with the ability to complete online payments for the services they select.

[1642] "Server" refers to a computer system that processes data received from a user's device and uses a generative AI model to analyze and make recommendations.

[1643] "Terminal" refers to a device used by a user to access the system, including smartphones and tablets.

[1644] An "HTTP request" refers to a communication protocol for sending data to a web server, and is used when sending data such as a photo of a person's face to a server.

[1645] "JSON format" is an abbreviation for JavaScript Object Notation, and refers to a format for structuring and describing data, and is used for sending proposal results, etc.

[1646] The present invention is a system that allows users to receive personalized beauty and fashion recommendations and smoothly make reservations and payments. Specific methods for implementing this system are described in detail below.

[1647] First, the user uses a device such as a smartphone or tablet. The user launches a dedicated application installed on the device and uploads a photo of their face through the application. Specifically, the user can take a new photo of their face using the camera function or select an existing photo from the gallery. The uploaded photo is temporarily stored on the device.

[1648] The device then sends the temporarily stored facial photo data to the server as an HTTP request. The server preprocesses the received facial photo data using an image processing algorithm and inputs this preprocessed data into the generative AI model. The generative AI model uses deep learning to convert the facial photo's features into numerical data and extract information such as contours, skin tone, and eye shape. The analysis results are stored in the server's internal database.

[1649] The server then retrieves the analysis results of the user's facial photo from its internal database and combines them with the preferences and current trends the user has entered through the application. Using this information, the generative AI model generates optimal makeup, hairstyle, and fashion recommendations. The generated recommendations are converted into JSON format and sent from the server to the device.

[1650] The user checks these suggestions on the device, selects their favorite hair salon, beauty salon, or fashion store, and makes a reservation. Specifically, the user enters the necessary information, such as the service content and desired date and time, into the reservation form within the application to confirm the reservation. The device sends the reservation information to the server, which then notifies the specified store of that information.

[1651] Finally, the user uses the payment platform to make an online payment. The terminal displays an interface for inputting payment information, and after the user inputs the required payment information, the payment is completed through the payment platform. The server receives a payment completion notification from the payment platform and sends the information to the terminal together with the reservation information.

[1652] For example, if a user is interested in natural makeup and a bob hairstyle, they can use the application to upload a photo of their face and input this information along with their desired style. The server uses a generative AI model to generate optimal makeup and hairstyle suggestions and presents them to the user. The user can then select a suggested salon, make a reservation, and complete payment online.

[1653] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1654] Program processing flow

[1655] Step 1:

[1656] Users launch the dedicated application on their smartphone, tablet, or other device, and can either take a new photo of their face using the camera function on the application's home screen, or select an existing photo from their gallery.

[1657] Input: User interaction via smartphone or tablet

[1658] Output: A face photo data to be uploaded

[1659] Step 2:

[1660] Users upload their facial photo data through the application interface, and by clicking the upload button, the facial photo data is temporarily saved on the device.

[1661] Input: Face photo data selected or taken by the user

[1662] Output: Temporarily saved face photo data

[1663] Step 3:

[1664] The device sends the temporarily stored facial photo data to the server in the form of an HTTP request. Specifically, it uses a REST API to send the data to the server in the form of multipart form data.

[1665] Input: Temporarily saved face photo data

[1666] Output: Face photo data sent to the server

[1667] Step 4:

[1668] The server inputs the received facial photo data into the generative AI model, checks the data format, and performs preprocessing if necessary before passing it to the AI ​​model.

[1669] Input: Face photo data received from the device

[1670] Output: Preprocessed facial photo data that is fed into the AI ​​model

[1671] Step 5:

[1672] The generative AI model analyzes facial photo data and extracts numerical data as features, such as facial contours, skin tone, and eye shape.

[1673] Input: Preprocessed facial photo data

[1674] Output: Extracted feature data

[1675] Step 6:

[1676] The server stores the feature data obtained from the generative AI model in an internal database, along with the beauty and fashion preferences and trend information entered by the user.

[1677] Input: Extracted feature data, user preferences and trend information

[1678] Output: Feature data and preference information stored in the internal database

[1679] Step 7:

[1680] The server retrieves feature data and preference information from an internal database, and then uses a generative AI model based on this to generate optimal makeup, hairstyle, and fashion suggestions.

[1681] Input: Feature data, user preferences and trend information

[1682] Output: Generated proposal data

[1683] Step 8:

[1684] The server converts the generated proposal data into JSON format and sends it to the device using a REST API, sending the encoded data packet to the device.

[1685] Input: Generated proposal data

[1686] Output: Proposal data in JSON format, data sent to the device

[1687] Step 9:

[1688] The device displays the received recommendation data to the user, who then checks the displayed recommendations and selects their preferred beauty salon, aesthetic salon, or fashion store.

[1689] Input: JSON formatted proposal data received from the server

[1690] Output: The suggestions displayed to the user

[1691] Step 10:

[1692] The user follows the suggestions, enters the necessary information (service details, date and time, etc.) into the reservation form within the application, and presses the reservation button to confirm the reservation.

[1693] Input: Booking information entered by the user

[1694] Output: Confirmed reservation information

[1695] Step 11:

[1696] The terminal sends the confirmed reservation information to the server, and the server notifies the store of the received reservation information.

[1697] Input: Reservation information confirmed by the user

[1698] Output: Reservation information sent to the server, reservation information notified to the target store

[1699] Step 12:

[1700] To make a payment using the payment platform, the user inputs the necessary payment information (such as card information) into the payment interface on the terminal, which then transmits this information to the payment platform for payment processing.

[1701] Input: User's payment information

[1702] Output: Payment information sent to the payment platform, payment processing results

[1703] Step 13:

[1704] The server receives a payment completion notification from the payment platform and transmits the information to the terminal together with the user's reservation information.

[1705] Input: Payment completion notification from payment platform

[1706] Output: Payment completion notification and reservation information sent to the user's terminal

[1707] Through these processing steps, users can receive personalized beauty and fashion recommendations using generative AI models, and make reservations and payments smoothly.

[1708] (Application example 1)

[1709] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1710] Systems that offer beauty and fashion recommendations are expected to enable users to quickly and accurately receive personalized recommendations, while also enabling smooth in-store reservations and payment. While conventional systems analyze users' facial photos to provide recommendations, these recommendations often do not adequately address individual preferences or trends. Furthermore, the reservation and payment processes are complicated, resulting in poor user convenience. Therefore, there is a need for a system that allows users to receive personalized recommendations and easily make reservations and payments.

[1711] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1712] In this invention, the server includes means for a user to upload a facial photo, means for transmitting the facial photo to the server, means for the server to analyze the facial photo using a generative AI model, means for suggesting makeup, hairstyles, and fashions that suit the user's preferences based on the analysis results obtained by the generative AI model, means for making a store reservation based on the suggested makeup, hairstyles, and fashions, means for making a payment through a payment platform, means for saving the user's data in an internal database, and interface means for checking the suggestion results on a smartphone. This allows the user to receive personalized beauty and fashion suggestions and smoothly make a reservation and make a payment at the suggested store.

[1713] "Means for users to upload facial photos" refers to the interface and functionality that allows users to select or take a photo of their face using a smartphone or other device and upload it into the application.

[1714] The "means for transmitting facial photographs to a server" refers to a communication protocol and interface for securely and efficiently transmitting uploaded facial photographs of users to a server.

[1715] "Means for the server to analyze facial photos using a generative AI model" refers to the function of inputting facial photo data received by the server into a generative AI model, which then analyzes facial features and extracts them as data.

[1716] "A means of suggesting makeup, hairstyles, and fashion that suit the user's preferences based on the analysis results obtained by a generative AI model" is a system that uses the results of facial photo analysis by a generative AI model to suggest optimal beauty and fashion options in combination with the user's preference information.

[1717] "Means for making store reservations based on suggested makeup, hairstyles, and fashion" refers to the interface and functionality for making reservations at hair salons, beauty salons, and fashion stores through the application based on the suggested results of the generative AI model.

[1718] "Means for making payments through a payment platform" means a payment system and interface for completing online payments for the booked services.

[1719] "Means for storing user data in an internal database" refers to a database system for securely storing user facial photographs and preference data for use in subsequent suggestions.

[1720] The "interface means for checking the proposal results on a smartphone" refers to a user interface that allows a user to visually check the beauty and fashion proposals from the generated AI model using a smartphone.

[1721] This invention is a system that allows users to receive personalized beauty and fashion suggestions, and smoothly make store reservations and payments. This system includes the following means.

[1722] A way for users to upload photos of their faces

[1723] The user starts the smartphone application, takes a photo of their face using the camera function or selects an existing photo from the gallery, and then uploads the photo data to the application through an interface for uploading a photo of their face.

[1724] A means of sending a face photo to the server

[1725] The device temporarily stores the uploaded facial photo data and prepares it for transmission to the server. The stored facial photo data is then sent to the server as an HTTP request. This is mainly done using the React Native HTTP request module.

[1726] The server generates a facial photo and analyzes it using an AI model.

[1727] The server inputs the facial photo data received from the device into the generative AI model. At this time, the server uses TensorFlow and PyTorch to analyze the facial photo's features (e.g., contours, skin tone, eye shape, etc.) and converts them into numerical data. The analysis results are then stored in an internal database (MongoDB) and transferred for subsequent processing.

[1728] How generative AI models generate proposals

[1729] The server retrieves the results of the user's facial photo analysis from an internal database, inputs the user's preferences and current trend information into the AI ​​model, and the generative AI model generates optimal suggestions for makeup, hairstyles, and fashion, and sends the generated suggestions in JSON format to the device.

[1730] A way to make restaurant reservations based on the proposed content

[1731] The user checks the suggestions and selects the hair salon, aesthetic salon, or fashion store that they like. They then enter reservation information, such as the service content and date and time of the selected store, and confirm the reservation.

[1732] A means of making payments through a payment platform

[1733] The terminal sends the reservation information entered by the user to the server. It then displays an interface for entering payment information, and the user enters the payment information. The entered payment information is sent to a payment platform (e.g., Stripe API) and the processing result is received. The server notifies the store of the reservation information received from the terminal, receives a payment completion notification from the payment platform, and confirms the completion of the reservation. The completed reservation and payment information are sent to the terminal.

[1734] Specific example explanation

[1735] For example, a user launches the "Beauty360" app on their smartphone and uploads a photo of their face. They then enter "I'm interested in natural makeup and bob hairstyles" through the interface. The server analyzes the photo and uses this information to generate optimal recommendations using a generative AI model. The user then confirms the recommendations, makes a reservation at the suggested salon, and makes payment via Stripe.

[1736] Example prompt sentence:

[1737] {

[1738] "face_features": {

[1739] "face_shape": "round",

[1740] "skin_tone": "fair",

[1741] "eye_shape": "almond"

[1742] },

[1743] "user_preference": {

[1744] "style": "natural",

[1745] "interest": ["hair", "makeup"]

[1746] },

[1747] "current_trends": {

[1748] "makeup": ["minimalist", "soft colors"],

[1749] "hair": ["long bob", "wavy"]

[1750] }

[1751] }

[1752] By writing and implementing it in this way, users will be able to receive personalized beauty and fashion suggestions, and easily make reservations and payments at physical stores using a smartphone application.

[1753] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1754] Step 1:

[1755] A user starts the smartphone application and takes a face photo using the camera function or selects an existing face photo from the gallery. This inputs face photo data. Next, the face photo data is uploaded to the application through the face photo upload interface and temporarily saved on the device.

[1756] Step 2:

[1757] The device sends the temporarily saved facial photo data to the server as an HTTP request. The server receives the facial photo data and prepares to start processing it as input data.

[1758] Step 3:

[1759] After receiving the facial photo data, the server uses TensorFlow to input the facial photo into a generative AI model. The generative AI model analyzes the facial photo's features (contours, skin tone, eye shape, etc.) and converts these features into numerical data. The converted data is stored in an internal database (e.g., MongoDB) and transferred for subsequent processing.

[1760] Step 4:

[1761] The server retrieves the results of the user's facial photo analysis from its internal database and inputs them into the generative AI model along with the user's preferences and trend information. The generative AI model then generates makeup, hairstyle, and fashion suggestions and outputs them in JSON format. The output suggestions are then sent to the device.

[1762] Step 5:

[1763] The user checks the recommendations through their smartphone application. At this time, the user interface visually displays the beauty recommendations created by the generative AI model. The user then selects their favorite beauty salon, beauty salon, or fashion store.

[1764] Step 6:

[1765] The user inputs the service details and date and time of the selected store and sends the reservation information to the server. The server notifies the corresponding store of the received reservation information. The notified store confirms the reservation and sends the reservation confirmation information to the server.

[1766] Step 7:

[1767] The terminal sends the reservation information entered by the user to the server, and then displays an interface for entering payment information. The user enters the payment information, and the terminal sends the information to a payment platform (e.g., Stripe API) for processing. The processing result is sent to the server.

[1768] Step 8:

[1769] The server receives a payment completion notification from the payment platform and confirms the completion of the reservation. This completion information is sent to the terminal so that the user can check it. This completes the entire process.

[1770] By following the above steps, users can receive personalized beauty and fashion recommendations and smoothly make reservations and payments. Examples of prompts are as follows:

[1771] {

[1772] "face_features": {

[1773] "face_shape": "round",

[1774] "skin_tone": "fair",

[1775] "eye_shape": "almond"

[1776] },

[1777] "user_preference": {

[1778] "style": "natural",

[1779] "interest": ["hair", "makeup"]

[1780] },

[1781] "current_trends": {

[1782] "makeup": ["minimalist", "soft colors"],

[1783] "hair": ["long bob", "wavy"]

[1784] }

[1785] }

[1786] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1787] The present invention is a system that provides beauty and fashion recommendations based on a user's facial photo and preferences, and allows for smooth reservations and payments. It also has the ability to recognize the user's emotional state and make optimal recommendations based on that. The following elements are required to implement this system:

[1788] A way for users to upload photos of their faces

[1789] User:

[1790] 1. Launch the application and take a photo of your face using the camera function or select an existing photo from your gallery.

[1791] 2. Upload your face photo data to the application through the interface for uploading face photos.

[1792] A means of sending a face photo to the server

[1793] Device:

[1794] 1. The uploaded facial photo data is temporarily saved and prepared for sending to the server.

[1795] 2. The saved facial photo data is sent to the server as an HTTP request.

[1796] The server generates a facial photo and analyzes it using an AI model.

[1797] server:

[1798] 1. The facial photo data received from the device is input into the generative AI model.

[1799] 2. The generative AI model analyzes the facial features (e.g., facial contours, skin tone, eye shape, etc.) and converts them into numerical data.

[1800] 3. The analysis results are stored in an internal database for subsequent processing.

[1801] Using an emotion engine to recognize user emotions

[1802] server:

[1803] 1. Input facial photo data into the emotion engine to analyze the user's emotional state (e.g., happiness, sadness, surprise, etc.).

[1804] 2. The analysis results of the emotion engine are stored in an internal database.

[1805] How generative AI models generate proposals

[1806] server:

[1807] 1. Obtain the analysis results and emotional state of the user's face photo from the internal database.

[1808] 2. User-input preferences and current trend information are fed into the AI ​​model to generate optimal makeup, hairstyle, and fashion suggestions.

[1809] 3. It also dynamically adjusts its suggestions based on your emotional state.

[1810] 4. The generated proposal results are sent to the device.

[1811] A way to make a reservation based on the suggestions

[1812] User:

[1813] 1. Check the suggestions and select your favorite hair salon, beauty salon, or fashion store.

[1814] 2. Enter reservation information such as the service content, date and time of the selected store and confirm your reservation.

[1815] A means of making payments through a payment platform

[1816] Device:

[1817] 1. The reservation information entered by the user is sent to the server.

[1818] 2. An interface for entering payment information is displayed, and the user enters the payment information.

[1819] 3. The entered payment information is sent to the payment platform and the processing result is received.

[1820] server:

[1821] 1. Notify beauty salons, esthetic salons, and fashion stores of reservation information received from the device.

[1822] 2. Receive a payment confirmation from the payment platform to confirm the completion of your booking.

[1823] 3. Send the completed booking and payment information to the terminal.

[1824] Specific examples

[1825] Example 1: Makeup and hairstyle suggestions

[1826] User:

[1827] 1. Upload a photo of your face using the application and indicate that you are interested in natural makeup and bob hairstyles.

[1828] 2. Check the suggestions from the server and make a reservation at the suggested beauty salon.

[1829] Device:

[1830] 1. Send a face photo and user preferences to the server.

[1831] 2. Receive suggestions from the server and present them to the user.

[1832] 3. Send the reservation information to the server and display the payment interface.

[1833] server:

[1834] 1. Analyze your facial photo and suggest the best makeup and hairstyle.

[1835] 2. The emotion engine analyzes the user's emotional state and adjusts the suggestions accordingly.

[1836] 3. Notify the beauty salon of the user's reservation information and receive payment completion information.

[1837] Example 2: Fashion coordination suggestions

[1838] User:

[1839] 1. Enter your interest in casual style and upload a photo of your face.

[1840] 2. Make an appointment at the suggested fashion store and complete the payment online.

[1841] Device:

[1842] 1. Send your face photo and preference information to the server.

[1843] 2. Receive suggestions from the server and present them to the user.

[1844] 3. Send the reservation information to the server and process the payment.

[1845] server:

[1846] 1. We suggest the best fashion coordination based on the analysis of your face photo and your preferences.

[1847] 2. The emotion engine analyzes the user's emotional state and adjusts the suggestions accordingly.

[1848] 3. The user's reservation information is notified to the fashion store and payment completion information is received.

[1849] As described above, the system of the present invention allows users to receive personalized beauty and fashion suggestions using generative AI models and emotion engines, and to easily make reservations and payments.

[1850] The processing flow will be explained below.

[1851] Step 1:

[1852] User: Launches the application and takes a photo of his / her face using the camera function or selects an existing photo from the gallery. Clicks the button to upload a photo of his / her face.

[1853] Step 2:

[1854] Terminal: The facial photo data selected or taken by the user is temporarily stored within the application. An HTTP request is generated to send the stored facial photo data to the server. The generated HTTP request is sent to the server.

[1855] Step 3:

[1856] Server: Receives facial photo data sent from the device. Preprocesses the facial photo data to pass it to the generative AI model and emotion engine.

[1857] Step 4:

[1858] Server: The generative AI model analyzes the facial features (e.g., facial contours, skin tone, eye shape, etc.) and converts them into numerical data. The analysis results are stored in an internal database.

[1859] Step 5:

[1860] Server: Inputs a facial photo into the emotion engine and analyzes the user's emotional state (e.g., happiness, sadness, surprise, etc.). The analysis results of the emotion engine are also stored in an internal database.

[1861] Step 6:

[1862] User: Fills out a form to select their makeup, hairstyle and fashion preferences; answers choices and questions about current trends; completes the form and clicks the "Submit" button.

[1863] Step 7:

[1864] Terminal: The application temporarily stores the preferences and trend information entered by the user. It generates an HTTP request to send the stored data to the server. It then sends the generated HTTP request to the server.

[1865] Step 8:

[1866] Server: Receives preferences and trend information sent by the user. Combining the results of facial photo analysis, emotional state, and preference information, the generative AI model generates optimal makeup, hairstyle, and fashion suggestions. The suggestions are dynamically adjusted based on the user's emotional state.

[1867] Step 9:

[1868] Server: Sends the proposed results from the generative AI model to the terminal.

[1869] Step 10:

[1870] Terminal: Receives the proposal results sent from the server. The received proposal results are displayed within the application and confirmed by the user.

[1871] Step 11:

[1872] User: Check the suggestions and select a hair salon, beauty salon, or fashion store that suits them best. Enter the reservation information (date, time, service details, etc.) for the selected store and click the "Confirm reservation" button.

[1873] Step 12:

[1874] Terminal: The reservation information entered by the user is temporarily saved within the application. An HTTP request is generated to send the reservation information to the server. The generated HTTP request is sent to the server.

[1875] Step 13:

[1876] Server: Receives reservation information sent from the terminal. Generates and sends another HTTP request to the store's system to notify the corresponding store of the received reservation information. Receives reservation confirmation or adjustment information from the store and stores it.

[1877] Step 14:

[1878] Server: Sends reservation confirmation information to the terminal.

[1879] Step 15:

[1880] Terminal: Receives the reservation confirmation information sent from the server and displays it to the user. Generates a link to the payment platform and presents it to the user.

[1881] Step 16:

[1882] User: Click on the payment platform link displayed on the terminal, select the payment method, enter the required payment information, and click the "Complete Payment" button.

[1883] Step 17:

[1884] Terminal: Sends the payment information entered by the user to the payment platform. Receives the payment completion notification and generates an HTTP request to send to the server. Sends the generated HTTP request to the server.

[1885] Step 18:

[1886] Server: Receives the payment completion notification sent from the terminal. Based on the received information, sends a reservation completion and payment completion notification to the relevant store. Sends a completion notification to the terminal.

[1887] Step 19:

[1888] Terminal: Receives the reservation and payment completion notification sent from the server and displays it to the user.

[1889] Example 2

[1890] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1891] Conventional beauty and fashion recommendation systems have difficulty providing personalized recommendations that comprehensively consider a user's facial photo, emotional state, preferences, etc. Furthermore, there has been a lack of systems that allow users to instantly complete reservations and payments based on the recommendations. Therefore, there has been a strong demand for the development of a system that allows users to easily receive optimal beauty and fashion recommendations and quickly use the service.

[1892] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1893] In this invention, the server includes means for using an emotion engine that analyzes the user's emotional state, means for adjusting the content of suggestions according to the emotional state, and means for suggesting makeup, hairstyles, and fashions that suit the user's preferences based on the analysis results obtained by the generative AI model. This makes it possible to quickly provide personalized and optimal suggestions by comprehensively considering the user's facial photo and emotional state, and to smoothly execute the reservation and payment processes.

[1894] A "user" is a person who uses the system to upload a photo of their face and receive beauty and fashion suggestions.

[1895] A "face photo" is still image data that includes features such as the user's facial contours, eye shape, and skin tone.

[1896] The "server" is a central processing unit that processes facial photo data, user preferences, and emotional state, and generates optimal suggestions using generative AI models and emotion engines.

[1897] A "generative AI model" is an artificial intelligence model that analyzes facial photos and converts their features into numerical data.

[1898] An "emotion engine" is software that analyzes a user's emotional state (e.g., happiness, sadness, surprise, etc.) from a photograph of their face.

[1899] "Makeup" refers to methods and products for applying makeup to a user's face.

[1900] A "hairstyle" is a method or design for instructing the shape and arrangement of a user's hair.

[1901] "Fashion" refers to a user's clothing, accessories, and other decorative items related to their appearance.

[1902] A "reservation" is a procedure in which a user determines in advance the date, time, and content of services to be used at a beauty salon, esthetic salon, or fashion store.

[1903] "Payment Platform" means an online payment system through which Users pay for Services.

[1904] "Suggestions" are specific advice and options regarding makeup, hairstyles, and fashion provided to users based on the results of analysis by generative AI models and emotion engines.

[1905] MODE FOR CARRYING OUT THE INVENTION

[1906] This system provides beauty and fashion recommendations based on a user's facial photograph and preferences, and allows for smooth booking and payment. It also has the ability to recognize the user's emotional state and provide optimal recommendations based on that. The specific configuration and operating procedures for implementing this system are described below.

[1907] A way for users to upload photos of their faces

[1908] The user launches the application on their smartphone or PC and takes a photo of their face using the camera function, or selects an existing photo from the gallery. Then, the user presses the upload button on the application to upload the photo data to the application. This application can run on the iOS or Android platform, for example.

[1909] A means of sending a face photo to the server

[1910] The device temporarily stores the uploaded facial photo data and sends it to the server in the form of an HTTP request. The HTTP request header contains authentication information, and the body contains the facial photo data. The device's local storage is used for temporary storage.

[1911] The server generates a facial photo and analyzes it using an AI model.

[1912] The server inputs the facial photo data received from the device into a generative AI model. The server performs preprocessing on the facial photo data, such as resizing and color normalization. The data is then input into a specialized facial analysis AI model (e.g., a convolutional neural network using TensorFlow) to convert features such as facial contours, skin tone, and eye shape into numerical data. The results of this analysis are stored in an internal database and used for subsequent processing.

[1913] Using an emotion engine to recognize user emotions

[1914] The server inputs the facial photo data into an emotion engine to analyze the user's emotional state (e.g., happiness, sadness, surprise, etc.). The emotion engine's analysis results are also stored in an internal database. The emotion engine uses, for example, facial expression recognition software (e.g., Microsoft's Azure Emotion API).

[1915] How generative AI models generate proposals

[1916] The server retrieves the analysis results of the user's facial photo and emotional state from an internal database, inputs the user's preferences and current trend information into the AI ​​model, and generates optimal suggestions for makeup, hairstyles, and fashion. Furthermore, the suggestions are dynamically adjusted according to the user's emotional state. The results of these suggestions are returned to the device in JSON format.

[1917] For example, if a user enters "I'm interested in natural makeup and bob hairstyles" and uploads a photo of their face, the server will use a generative AI model to analyze the user's facial features and make optimal suggestions for natural makeup and bob hairstyles based on those features.

[1918] Example prompt sentence:

[1919] "I'm interested in natural makeup and bob hairstyles."

[1920] A way to make a reservation based on the suggestions

[1921] The user reviews the suggestions and selects their favorite beauty salon, esthetic salon, or fashion store on the application. They then enter reservation information such as the service content and date and time of the selected store and press "Confirm reservation" to make the reservation. The reservation information is then sent from the device to the server.

[1922] A means of making payments through a payment platform

[1923] The terminal sends the reservation information entered by the user to the server, and also displays an interface for entering payment information, where the user enters payment information such as credit card information. The terminal sends data to the API of the payment platform (e.g., Stripe or PayPal) and receives the processing results.

[1924] A means by which the server processes booking and payment information

[1925] The server notifies the beauty salon, esthetic salon, or fashion store of the reservation information received from the device. It also receives a payment completion notification from the payment platform and confirms the completion of the reservation. The completed reservation and payment information are returned to the device in JSON format.

[1926] As described above, the system of the present invention allows users to receive personalized beauty and fashion suggestions using generative AI models and emotion engines, and to easily make reservations and payments.

[1927] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1928] Step 1:

[1929] Users launch the application on their smartphone or PC, take a photo of their face using the camera function, or select an existing photo from the gallery, and then press the upload button to upload the photo data to the application.

[1930] Input: Facial photo data taken or selected by the user

[1931] Output: Facial photo data sent to the application through the upload interface

[1932] Specific actions: The user taps the app's camera icon to launch the camera, takes a photo of their face, and taps "Done." Alternatively, the user taps the gallery icon, selects an existing photo, and presses the "Upload" button.

[1933] Step 2:

[1934] The device temporarily stores the uploaded facial photo data and sends it to the server in the form of an HTTP request.

[1935] Input: Uploaded face photo data

[1936] Output: Facial photo data sent to the server in the form of an HTTP request

[1937] Specific operation: The device temporarily stores the face photo data in local storage and then sends it to the server's API endpoint in the background. The HTTP request header contains authentication information, and the body contains the face photo data.

[1938] Step 3:

[1939] The server inputs the facial photo data received from the device into the generative AI model. The server preprocesses the facial photo data by resizing and color normalizing it, and then inputs the data into the generative AI model.

[1940] Input: Facial photo data received from the device

[1941] Output: Analysis results converted from facial features into numerical data

[1942] Specific operation: The server performs preprocessing on the facial photo data, such as resizing and color normalizing, and then inputs the data into a generative AI model (e.g., a convolutional neural network using TensorFlow), converting features such as facial contours, skin tone, and eye shape into numerical data.

[1943] Step 4:

[1944] The server inputs the facial photo data into an emotion engine to analyze the user's emotional state (e.g., happiness, sadness, surprise, etc.).

[1945] Input: Facial photo data

[1946] Output: Analysis results showing the user's emotional state

[1947] Specific operation: The server inputs facial photo data into an emotion engine (e.g., facial expression recognition software), analyzes the emotional state, and stores the results in an internal database.

[1948] Step 5:

[1949] The server retrieves the analysis results of the user's facial photo and emotional state from an internal database, and inputs the user's input preferences and current trend information into an AI model to generate optimal suggestions for makeup, hairstyles, and fashion.

[1950] Input: User's facial photo analysis results, emotional state, preferences and trend information

[1951] Output: Makeup, hairstyle, and fashion suggestions

[1952] Specific operation: The server retrieves the necessary data from the internal database, integrates them, and inputs them into the generative AI model. The generative AI model generates a proposal, adjusts the proposal content using the emotion engine, and sends it to the device in JSON format.

[1953] Step 6:

[1954] The user checks the suggestions on the application and selects the hair salon, beauty salon, or fashion store that suits them best. After that, they enter reservation information such as the service content, date and time, and confirm the reservation.

[1955] Input: Proposal details, reservation information (service details, date and time)

[1956] Output: Confirmed reservation information

[1957] Specific operation: The user checks the list of suggestions, taps the selection button to move to the reservation page, enters the service details and date and time, and taps "Confirm reservation."

[1958] Step 7:

[1959] The terminal transmits the reservation information entered by the user to the server, displays an interface for entering payment information, and the user enters the payment information. The terminal then transmits the data to the payment platform and receives the processing result.

[1960] Input: Reservation information, payment information

[1961] Output: Processing result by payment platform

[1962] Specific operation: The terminal sends the reservation information to the server, then displays the payment page. The user enters credit card information and taps the "Pay" button. The terminal then sends the data to the payment platform's API and receives the processing result.

[1963] Step 8:

[1964] The server notifies the beauty salon, esthetic salon, or fashion store of the reservation information received from the terminal, receives a payment completion notification from the payment platform, confirms the completion of the reservation, and sends the completed reservation and payment information to the terminal.

[1965] Input: Reservation information, payment completion notification

[1966] Output: Reservation completion information, payment completion information

[1967] Specific operation: The server notifies the store management system of the reservation information in real time, then receives a payment completion notification from the payment platform, updates the reservation status to "Completed", and returns the reservation and payment completion information to the terminal in JSON format.

[1968] (Application example 2)

[1969] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1970] Conventional beauty and fashion recommendation systems were unable to provide personalized recommendations that fully considered the individual characteristics and emotional state of each user. As a result, users were often dissatisfied with the recommendations and stopped using the service. In addition, the reservation and payment procedures were cumbersome, leaving a need for an improved user experience.

[1971] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1972] In this invention, the server includes means for a user to upload a facial photo, means for transmitting the facial photo to the server, means for the server to use a generative AI model that analyzes the facial photo, means for suggesting beauty and fashion that matches the user's preferences based on the analysis results obtained by the generative AI model, means for making a reservation at a service facility based on the suggested beauty and fashion, means for making a payment through a payment platform, means for inputting the user's facial photo and preference information through a smartphone application, means for recognizing the user's emotional state using an emotion engine, and means for dynamically adjusting the content of the proposals according to the emotional state. This not only enables personalized proposals based on the user's individual characteristics and emotional state, but also enables a smooth process from reservation to payment.

[1973] "Means for users to upload facial photos" refers to the methods or functions that allow users to take or select their own facial photos and transmit the photo data via the Internet.

[1974] "Means for sending facial photos to a server" refers to a system that has the function of temporarily storing facial photos taken or selected by a user and sending them to an external server via the Internet.

[1975] "Means of using generative AI models" refers to methods or functions that use models to generate results by using artificial intelligence to analyze collected data.

[1976] "Means of suggesting beauty and fashion that suits the user's preferences" refers to a function that suggests beauty, hairstyles, and fashion that suits the user's individual preferences based on the analysis results obtained by the generative AI model.

[1977] "Means for making reservations at service providing facilities" refers to a function that enables users to make online reservations at service providing facilities such as beauty salons, esthetic salons, and fashion stores based on the beauty and fashion recommendations.

[1978] "Means of making payments through a payment platform" refers to the use of payment systems and services to make payments for online reservations in a secure and efficient manner.

[1979] "Means for inputting a user's facial photograph and preferred information via a smartphone application" refers to an interface that allows a user to input a photograph of their face and preferred beauty and fashion information via an application that runs on a smartphone.

[1980] "Means for recognizing a user's emotional state using an emotion engine" refers to a method that uses a system or algorithm to analyze a facial photograph and determine the user's current emotional state (such as joy, sadness, surprise, etc.).

[1981] "Means for dynamically adjusting the content of suggestions according to the emotional state" refers to a function that appropriately changes and adjusts the content of beauty and fashion suggestions in real time based on the recognized emotional state of the user.

[1982] The present invention is a system in which a user uploads a photo of their face via a smartphone application, and the server analyzes the photo using a generated AI model to suggest beauty and fashion products that match the user's preferences. This system is realized through the following configuration and processing.

[1983] The server first receives a facial photo taken or selected by the user using a smartphone application. The facial photo is uploaded using the smartphone's camera or gallery function. This photo data is then sent to the server via the Internet.

[1984] The server temporarily stores the received facial photo data and analyzes it using a generative AI model. The analysis extracts facial features such as facial contours, skin tone, and eye shape as numerical data. This process uses machine learning frameworks such as TensorFlow.

[1985] Furthermore, the server uses an emotion engine to recognize the user's emotional state. The emotion engine can determine the user's current emotion (e.g., joy, sadness, surprise, etc.) from the facial photograph. This information is also stored in an internal database as numerical data.

[1986] The server uses a generative AI model to generate optimal beauty and fashion recommendations based on the user's preferences, current trends, and the acquired facial features and emotional state. The recommendations are dynamically adjusted according to the user's emotional state and presented in a form optimized for each individual user. The recommendations are displayed to the user via a smartphone application.

[1987] Users can check the suggestions and make reservations at beauty salons, esthetic salons, and fashion stores that they are interested in. They enter reservation information through the smartphone application and confirm the reservation. The reservation information is sent to the server, which then notifies the relevant service provider.

[1988] Finally, payment is processed. The smartphone application connects to a payment platform and securely processes the payment information entered by the user. Services such as Stripe can be used as payment platforms. A notification of payment completion is sent to the server, confirming the completion of the reservation.

[1989] As a concrete example, consider the following prompt sentence:

[1990] Example prompt sentence:

[1991] "User's face photo has been uploaded and preprocessed. Generate a personalized beauty or fashion suggestion based on the following parameters:

[1992] Face features: [numerical data such as face shape, skin tone, eye shape, etc.]

[1993] Emotion state: "Joy"

[1994] User preferences: ["Natural makeup", "Bob hairstyle"]

[1995] Current trends: ["Spring 2023 trends"]

[1996] This will enable users to receive beauty and fashion suggestions based on their facial photos, preferences, and real-time emotional state, and will also enable them to complete reservations and payments on the spot.

[1997] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1998] Step 1:

[1999] The user starts the smartphone application and takes a photo of their face or selects one from the gallery. They then use the application's camera or gallery function to obtain a photo of their face and press the upload button to enter the photo data. The entered photo data is temporarily saved.

[2000] Step 2:

[2001] The device sends face photo data to the server. The face photo data is temporarily saved and then sent to the server via an HTTP request. The input in this step is the face photo data, and the output is the data sent to the server.

[2002] Step 3:

[2003] The server inputs the received facial photo data into a generative AI model for analysis. Using a machine learning framework such as TensorFlow, facial photo features (contours, skin tone, eye shape, etc.) are extracted as numerical data. The input is facial photo data, and the output is numerical data indicating facial features.

[2004] Step 4:

[2005] The server uses an emotion engine to recognize the user's emotional state. The received facial photo data is input into the emotion engine to determine the user's emotional state (e.g., joy, sadness, surprise, etc.). The emotion engine's specific operation uses an algorithm to analyze facial expressions and micro-expressions. The input is facial photo data, and the output is data indicating the user's emotional state.

[2006] Step 5:

[2007] The server generates beauty and fashion suggestions using a generative AI model based on facial feature data, emotional state data, user preference information, and trend information. This uses prompts that take into account the user's input preferences (e.g., natural makeup, bob hairstyle) and current trend information. Specific examples of generated prompts include:

[2008] "User's face photo has been uploaded and preprocessed. Generate a personalized beauty or fashion suggestion based on the following parameters:

[2009] Face features: [numerical data such as face shape, skin tone, eye shape, etc.]

[2010] Emotion state: "Joy"

[2011] User preferences: ["Natural makeup", "Bob hairstyle"]

[2012] Current trends: ["Spring 2023 trends"]

[2013] The inputs are facial feature data, emotional state data, user preference information, and trend information, and the output is suggested beauty and fashion content.

[2014] Step 6:

[2015] The user checks the proposed beauty and fashion details on a smartphone application and selects the service provider of interest (beauty salon, esthetic salon, fashion store). The user then enters the reservation information for the desired service (date, time, location, service details, etc.) and presses the reservation button. The input is the confirmation result of the proposal details and the reservation information, and the output is the entered reservation information.

[2016] Step 7:

[2017] The terminal sends the entered reservation information to the server, which then notifies the relevant service provider of that information. The input is the reservation information, and the output is a notification to the service provider.

[2018] Step 8:

[2019] A user enters payment information into a smartphone application to make a payment. The application then sends the entered payment information to a payment platform (e.g., Stripe) to process the payment. The input is the payment information, and the output is the payment processing result.

[2020] Step 9:

[2021] The server receives the payment completion notification and confirms the completion of the reservation. It then notifies the user of the completed reservation information and payment information. The input is the payment processing result, and the output is the completed reservation information and payment completion notification.

[2022] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the control target 443 to output the result of the specific processing....

Claims

1. a means for a user to upload a photograph of their face; A means for transmitting a facial photograph to a server; A means for using a generative AI model in which the server analyzes facial photographs; A means to suggest makeup, hairstyles, and fashion that suit the user's preferences based on the analysis results obtained by the generative AI model, and A means to make reservations at beauty salons, esthetic salons, and fashion stores based on suggested makeup, hairstyles, and fashion. a means of making payments through a payment platform; A system including:

2. The system according to claim 1, further comprising means for inputting user preferences and trend information and allowing the generating AI model to make optimal suggestions based thereon.

3. 2. The system according to claim 1, further comprising means for the server to notify the store of reservation information for a beauty salon, esthetic salon or fashion store, and to receive reservation confirmation information.

Images