System

Abstract

A system is provided.SOLUTION: A system comprising: means for inputting basic information of a user; means for inputting a face photograph of the user and desired beauty style information; means including an artificial intelligence algorithm for analyzing the input information and making an optimal fashion proposal, makeup proposal, and hairstyle proposal; means for providing purchase source information of the proposed fashion and beauty item and information of a beauty salon; and means for providing appropriate advice based on the user's skin type or hair trouble and a list of related products.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] Today's consumers have high expectations for a wide variety of fashion and beauty options, yet struggle to find the style that best suits them. Furthermore, there is a lack of services that consistently offer comprehensive fashion and beauty recommendations tailored to each consumer's needs, making it time-consuming and labor-intensive to create their own style. The hassle of individually searching for information on where to purchase recommended items and beauty salons also needs to be eliminated. Furthermore, there is a lack of support that provides appropriate advice tailored to individual skin types and hair concerns. [Means for solving the problem]

[0005] The present invention provides a means for inputting a user's basic information, a facial photo, and desired beauty style information. It also provides a means including an artificial intelligence algorithm for analyzing the input information and providing optimal fashion, makeup, and hairstyle suggestions. Furthermore, it provides a means for providing information on where to purchase the suggested fashion and beauty items and beauty salons, and provides appropriate advice and a list of related products based on the user's skin type or hair concerns, thereby solving the above-mentioned problems. It also includes a means for redirecting users who wish to make a purchase to the suggested retailer or beauty salon, and a means for recording and saving users' economic behavior data, providing an environment in which users can easily take action.

[0006] "Basic information of the user" refers to data such as the user's age, gender, height, weight, skin color, budget, etc.

[0007] "Facial photo" refers to image data showing the user's appearance.

[0008] "Beauty style information" refers to detailed information about the makeup and hairstyle desired by the user.

[0009] "Artificial intelligence algorithm" refers to an algorithm that analyzes data entered by users and generates optimal fashion, makeup, and hairstyle suggestions.

[0010] "Fashion suggestions" refers to styling suggestions such as recommended clothing and accessories for users.

[0011] "Makeup suggestions" refer to suggestions regarding makeup methods and cosmetics recommended to users.

[0012] "Hairstyle suggestions" refers to suggestions regarding hairstyles and hair styling recommended to users.

[0013] "Where to Buy" refers to information about retail stores and online shops where suggested items can be purchased.

[0014] "Beauty salon information" refers to information about beauty salons where the proposed makeup and hairstyle can be applied.

[0015] "Skin type or hair concerns" refers to information about the user's skin condition or hair problems (dry skin, oily skin, dandruff, damaged hair, etc.).

[0016] "Advice" refers to suggestions for specific measures and improvements based on the user's skin type and hair concerns.

[0017] A "list of related products" refers to a list of recommended products to address a user's concerns.

[0018] "Redirect to purchase" refers to a method that transfers a user to the website or reservation system of the recommended item or salon appointment.

[0019] "Economic behavior data" refers to historical information about purchases and service usage made by users. [Brief explanation of the drawings]

[0020] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7]FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 3 is a sequence diagram showing a processing flow of the data processing system according to the first embodiment. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0021] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0022] First, the terms used in the following description will be explained.

[0023] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0024] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0025] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0026] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0027] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0028] [First embodiment]

[0029] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0030] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0031] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0032] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0033] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0034] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0035] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0036] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0037] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0038] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0039] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0040] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0041] The present invention relates to a system that utilizes artificial intelligence to provide a means for users to input their basic information, a photograph of their face, and information about their desired beauty style, and then analyzes this information to provide comprehensive fashion, makeup, and hairstyle suggestions. This system automatically provides optimal fashion and beauty suggestions by having users input the necessary information.

[0042] System configuration and operation

[0043] 1. Enter and submit user information

[0044] The user launches the application and enters basic information such as age, gender, height, weight, skin tone, budget, etc. The device packages the entered information in JSON format and sends it to the server.

[0045] 2. Register your face photo and beauty style

[0046] The user then uploads a photo of their face and selects the desired makeup and hairstyle, which is also compiled into JSON format by the device and sent to the server.

[0047] 3. AI-generated recommendations

[0048] Based on the received information, the server uses an artificial intelligence algorithm to comprehensively produce the most suitable fashion, makeup, and hairstyle for the user. This information is analyzed and an integrated visual proposal is generated. The proposal results are then sent from the server to the device.

[0049] 4. Review the proposal and view more information

[0050] The device receives the recommendations from the server and displays them to the user on the app interface. The user can review the recommendations and, if they like them, click the "Display details" button. This will display information about where to purchase the recommended fashion and beauty items and beauty salons.

[0051] 5. Purchase and reservation assistance

[0052] If the user wishes to purchase the suggested item or make a salon reservation, they operate the device and click on the link, which redirects the user to the purchase website or salon reservation system. The server records this action and stores it in the economic behavior database.

[0053] 6. Providing advice on skin type and hair problems

[0054] Additionally, users can input their skin type and hair concerns. The device sends this information to the server, which uses AI to generate appropriate advice and a list of related products and send them to the device. The device then displays the received advice and product list to the user.

[0055] Specific examples

[0056] Example 1: Entering basic information and beauty information

[0057] A 35-year-old female user logs into the app and enters her age, gender, height 160cm, weight 55kg, skin color light beige, and budget of 20,000 yen. Next, she uploads a photo of her face and selects her desired style of natural makeup and bob cut. The device compiles this information into JSON format and sends it to the server.

[0058] Example 2: Proposal generation and display

[0059] The server analyzes the input information and generates style suggestions, such as autumn casual fashion, natural makeup, and bob cuts, suitable for women of the same age. These suggestions are sent as visuals to the device. The device displays the suggestions to the user, who can click a button to view more information.

[0060] Example 3: Purchasing and reservation assistance

[0061] The user likes the suggested items, a blouse and a skirt, and clicks on a link to purchase them. The device redirects the user to the website where the items are purchased, and the server records this action.

[0062] In this way, the system based on the present invention provides users with quick and accurate comprehensive fashion and beauty suggestions, and smoothly links these to purchases and service use.

[0063] The processing flow will be explained below.

[0064] Program processing steps

[0065] Step 1: Enter your user information

[0066] Behavior 1.1

[0067] Users launch the app and enter basic information such as age, gender, height, weight, skin color, and budget into a form within the app.

[0068] Behavior 1.2

[0069] The device packages the basic information entered in JSON format and sends it to the server.

[0070] Step 2: Register your face photo and enter your beauty style

[0071] Behavior 2.1

[0072] Users can upload a photo of their face to the app using their device's camera or by selecting a suitable photo from their gallery.

[0073] Behavior 2.2

[0074] The user selects the desired makeup and hairstyle (e.g., natural makeup, bob cut).

[0075] Behavior 2.3

[0076] The device compiles the facial photo and beauty style information into JSON format and sends it to the server.

[0077] Step 3: Comprehensive production by AI

[0078] Behavior 3.1

[0079] The server then calls an artificial intelligence algorithm to analyze the received basic information, facial photo, and desired beauty style information.

[0080] Behavior 3.2

[0081] The server's artificial intelligence algorithm generates optimal fashion, makeup, and hairstyle suggestions based on the user's age, gender, height, weight, skin color, budget, and desired style information.

[0082] Behavior 3.3

[0083] The server packages the generated proposal results in JSON format and sends them to the terminal.

[0084] Step 4: Review your proposal and provide further information

[0085] Behavior 4.1

[0086] The device analyzes the suggestions received from the server and displays visual suggestions on the app interface.

[0087] Behavior 4.2

[0088] The user reviews the suggestions and, if they like them, clicks the button to display more information.

[0089] Behavior 4.3

[0090] The device displays a list of suggested fashion and beauty items, as well as information on where to purchase them and beauty salons.

[0091] Step 5: Purchase and reservation assistance

[0092] Operation 5.1

[0093] If the user wishes to purchase the suggested item or make a salon appointment, they click on the displayed link.

[0094] Behavior 5.2

[0095] Based on the link clicked, the device redirects the user to the purchase website or beauty salon reservation system.

[0096] Operation 5.3

[0097] The server records the user's purchasing and reservation behavior and stores it in an economic behavior database.

[0098] Step 6: Providing advice on skin type and hair concerns

[0099] Operation 6.1

[0100] Users can also enter their skin type and hair concerns (e.g., dry skin, dandruff, etc.) into the app.

[0101] Operation 6.2

[0102] The device packages the entered information about skin type and hair concerns in JSON format and sends it to the server.

[0103] Operation 6.3

[0104] The server uses artificial intelligence algorithms to generate appropriate advice and a list of related products based on the input skin type and hair concerns.

[0105] Operation 6.4

[0106] The server packages the generated advice and product list in JSON format and sends it to the terminal.

[0107] Operation 6.5

[0108] The device displays the received advice and a list of related products on the app interface.

[0109] ---

[0110] Through these steps, users can receive customized fashion and beauty suggestions through a consistent process, and can quickly make purchases or reservations.

[0111] Example 1

[0112] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0113] Conventional beauty and fashion recommendation systems have the problem that it takes a lot of time and effort for users to match their own information and preferences in detail. In addition, there are few systems that integrate various information to make comprehensive recommendations, and the purchasing and reservation process after the recommendations is often fragmented. This has led to issues such as low user satisfaction and a lack of improvement in system usage efficiency.

[0114] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0115] In this invention, the server includes means for inputting the user's personal information, means for inputting the user's face photo and desired beauty style information, means for packaging the input information into a data format and sending it to the server, means for analyzing the input information using an artificial intelligence algorithm and making optimal fashion, makeup, and hairstyle suggestions, means for displaying the suggestions received from the server to the user, means for providing information on where to purchase the suggested fashion and beauty items and information on beauty salons, and means for providing appropriate advice and a list of related products based on the user's skin type or hair concerns. This allows users to input their own information within a single system, receive optimal fashion and beauty suggestions based on that information, and also enable them to purchase related items and make beauty salon reservations in an integrated manner.

[0116] "User" means an individual who uses the System.

[0117] "Personal information" refers to information such as age, gender, height, weight, skin color, and budget that users enter into the system.

[0118] "Facial photo" refers to image data of a user's face that the user uploads to the system.

[0119] "Beauty style information" refers to information about the makeup and hairstyle style desired by the user.

[0120] A "data format" refers to a format that structures information to make it easier to store and communicate; a commonly used example is the JSON format.

[0121] "Server" refers to the central computer of the system, a computing resource that receives user input, analyzes it using artificial intelligence algorithms, and makes suggestions.

[0122] "Artificial intelligence algorithm" refers to machine learning and deep learning models and methods used to analyze input data and generate optimal fashion, makeup, and hairstyle suggestions.

[0123] "Suggestion results" is a general term for fashion suggestions, makeup suggestions, and hairstyle suggestions that are analyzed by an artificial intelligence algorithm and provided to users.

[0124] "Purchaser Information" refers to information about stores and online stores where suggested fashion and beauty items can be purchased.

[0125] "Beauty salon information" refers to information such as the location and contact details of the beauty salon that offers the proposed beauty style.

[0126] "Skin type" refers to the condition or characteristics of the user's skin (e.g., dry skin, oily skin).

[0127] "Hair concerns" refers to problems with the user's hair or areas they would like to improve (e.g., thinning hair, split ends).

[0128] The "List of Related Products" refers to a list of beauty products recommended based on the user's skin type and hair concerns.

[0129] "Purchase Intent" refers to a state in which a user has indicated an intention to purchase or use a proposed item or service.

[0130] "Redirecting" refers to the act of sending a user to another web page or service.

[0131] "Economic behavior data" refers to behavioral data when users engage in economic activities such as purchases and reservations.

[0132] "Recording and storing" refers to keeping specific information in a database or storage so that it can be accessed later.

[0133] The present invention relates to a system that utilizes artificial intelligence to input a user's basic information, a facial photo, and desired beauty style information, and then analyzes the information to provide comprehensive fashion, makeup, and hairstyle suggestions. This system automatically provides optimal fashion and beauty suggestions by having the user input the necessary information. Specific embodiments are described in detail below.

[0134] Enter and submit user information

[0135] Users launch a dedicated application and enter their personal information, including age, gender, height, weight, skin color, and budget. This information is packaged in JSON format by the device and sent to the server. Users also upload a photo of their face and select their desired beauty style (e.g., natural makeup, bob cut). This information is also packaged in JSON format by the device and sent to the server.

[0136] AI-powered recommendation generation

[0137] The server uses an artificial intelligence algorithm to analyze the received user information. This algorithm uses machine learning models such as TENSORFLOW (registered trademark) and PyTorch. The server generates optimal fashion, makeup, and hairstyle suggestions for the user, renders them in a visual format, and sends them to the device. The analyzed and generated suggestions are then sent to the device as JSON-formatted data.

[0138] Displaying the proposed results and providing detailed information

[0139] The device visually displays the received suggestions on the dedicated application interface. Users can check the suggestions displayed and click the confirmation button for more information, which will display information on where to purchase the suggested fashion items and detailed information on beauty salons.

[0140] Purchase and reservation assistance

[0141] If the user wishes to purchase the suggested item or make a reservation at a beauty salon, they click on the provided link on their device. By clicking on the link, the device redirects the user to the product purchase site or beauty salon reservation system. The user's clicking behavior is recorded by the server and later stored in the economic behavior database.

[0142] Advice on skin and hair problems

[0143] Additionally, users can enter additional information about their skin type and hair concerns. The device sends this information to the server, which uses AI algorithms to generate a list of appropriate advice and related products. This list and advice is then sent to the device and displayed to the user.

[0144] Examples and prompts

[0145] Example 1: Entering basic information and beauty information

[0146] A 35-year-old female user logs into the app and enters her age, height 160cm, weight 55kg, skin color light beige, and budget of 20,000 yen. Next, she uploads a photo of her face and selects her desired style of natural makeup and bob cut. The device compiles this information into JSON format and sends it to the server.

[0147] Example 2: Proposal generation and display

[0148] The server analyzes the input information and generates recommendations for autumn casual fashion, natural makeup, and a bob cut that are ideal for a 35-year-old woman, and sends them to the device. The device displays the recommendations and provides more information when the user clicks the "More Info" button.

[0149] Prompt Sentence Examples

[0150] "35-year-old female, height 160cm, weight 55kg, skin tone light beige, budget 20,000 yen. Prefer natural makeup and a bob cut. Please provide suggestions for casual fall fashion and specific shopping information."

[0151] The above is a detailed description of the embodiment of the present invention. This system allows users to efficiently and easily receive suggestions for optimal fashion and beauty styles, and then smoothly take the next action, such as purchasing a product or making a reservation at a beauty salon, based on that information.

[0152] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0153] Step 1: Enter and submit user information

[0154] The user starts the application and enters personal information (age, sex, height, weight, skin color, budget, etc.) by typing this information into a dedicated form.

[0155] Input: Personal information entered by the user

[0156] The terminal packages the entered information into JSON format.

[0157] Data processing: Validate the information entered by the user and convert it to JSON format.

[0158] Output: User details in JSON format

[0159] The terminal sends the generated JSON data to the server via an HTTP POST request.

[0160] Step 2: Register your face photo and beauty style

[0161] Users upload a photo of their face and select the desired beauty style (e.g., natural makeup, bob cut). Specifically, they take a photo of their face using the camera function on their smartphone and upload it to the app.

[0162] Input: Uploaded face photo and selected beauty style information

[0163] The device then reassembles this information into JSON format.

[0164] Data processing: Convert the face photo into the appropriate image format (e.g., JPEG) and include beauty style information in the JSON data.

[0165] Output: JSON formatted face photo and beauty style information

[0166] The terminal sends the generated JSON data to the server via an HTTP POST request.

[0167] Step 3: Generate recommendations with AI

[0168] The server analyzes the received JSON-formatted user information, face photo, and beauty style information using an artificial intelligence model (e.g., TensorFlow, PyTorch).

[0169] Input: JSON format user information, face photo, beauty style information

[0170] The server inputs this information into a generative AI model to generate optimal fashion, makeup, and hairstyle suggestions.

[0171] Data calculation: Generative AI models analyze input information and generate optimal recommendations

[0172] Output: JSON format suggestion results (fashion suggestions, makeup suggestions, hairstyle suggestions)

[0173] The server renders the generated proposal results in a visual format and sends them to the terminal as JSON data.

[0174] Step 4: View the results of the proposal and provide more information

[0175] The device displays the received proposal results on the interface of a dedicated app. The proposals are visually easy to understand.

[0176] Input: JSON formatted proposal results

[0177] Output: Visual suggestion results shown to the user

[0178] The user checks the suggested content and clicks the "Display detailed information" button, which displays information about where to purchase the suggested fashion items and detailed information about beauty salons.

[0179] Step 5: Assist with purchases and reservations

[0180] If users want to purchase or book a featured item or beauty service, they click on a link within the app.

[0181] Input: User clicks

[0182] The terminal responds to the click and redirects the user to a purchasing website or a beauty salon reservation system.

[0183] Data calculation: Link click event and redirect handling

[0184] Output: The redirected web page

[0185] The server records the user's clicking behavior and stores it in an economic behavior database.

[0186] Step 6: Providing advice on skin type and hair concerns

[0187] Users can also enter their skin type and hair concerns (e.g., dry skin, thinning hair).

[0188] Input: Information about your skin type and hair concerns

[0189] The device compiles this information into JSON format and sends it to the server.

[0190] The server passes the received information to an AI algorithm to generate optimal advice and related product lists.

[0191] Data arithmetic: AI algorithms analyze information and generate advice

[0192] Output: JSON formatted advice and related product list

[0193] The terminal displays the received information again to the user, allowing the user to check appropriate advice and related products.

[0194] (Application example 1)

[0195] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0196] Conventional fashion and beauty recommendation systems often require users to input information online and wait for a long time before receiving recommendations. They are also difficult to use in physical stores, lacking real-time support for users to try on clothes and check makeup looks. Furthermore, they lack the means to provide personalized advice based on a user's skin type or hair concerns. Therefore, there was a need for a system that provides optimal fashion and beauty recommendations in real time in physical stores and supports users in making their choices.

[0197] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0198] In this invention, the server includes means for inputting basic information about a user, means for inputting a face photo and desired beauty style information about the user, means including an artificial intelligence algorithm for analyzing the input information and making optimal fashion, makeup, and hairstyle suggestions, means for providing information on where to purchase the suggested fashion and beauty items and beauty salon information, means for providing appropriate advice and a list of related products based on the user's skin type or hair concerns, and means for receiving real-time fashion and beauty suggestions in a physical store via a smart device. This allows users to receive optimal fashion and beauty suggestions in real time in a physical store, enabling them to efficiently and effectively choose fashion and decide on a beauty style.

[0199] "Basic information" refers to information about a user's personal characteristics and attributes, such as age, sex, height, weight, skin color, and budget.

[0200] A "face photo" is image data of a photograph of a user's face.

[0201] "Beauty style information" is information about the makeup and hairstyle style desired by the user.

[0202] An "artificial intelligence algorithm" is a calculation method that analyzes input information and automatically generates optimal fashion, makeup, and hairstyle suggestions for users.

[0203] "Purchase Information" refers to information about locations or web pages where suggested fashion and beauty items can be purchased.

[0204] "Beauty salon information" refers to information about the location of stores that offer beauty services and how to make reservations.

[0205] "Skin type" refers to the type and characteristics of the user's skin, such as oily skin, dry skin, or sensitive skin.

[0206] "Hair problems" refers to problems or concerns that users have with their hair, such as hair loss, split ends, and dandruff.

[0207] "Appropriate advice" refers to solutions and care methods provided based on the user's skin type and hair concerns.

[0208] "Related Product Listings" refers to a list of products or services related to the relevant advice.

[0209] "Smart devices" refers to portable information devices that can connect to the Internet, such as smartphones and smart glasses.

[0210] "In real time in the physical store" means that suggestions and information are provided instantly on the spot while the customer is in the store.

[0211] To implement this invention, a system is required in which users, terminals, and servers each play a specific role.

[0212] Entering user information

[0213] Users use devices such as smartphones or smart glasses to enter basic information such as age, gender, height, weight, skin tone, and budget, which is collected through applications installed on the devices.

[0214] Register your face photo and beauty style

[0215] Next, the user captures a photo of their face and inputs their desired beauty style information. For example, they can select specific makeup and hairstyles, such as natural makeup or a bob cut. This photo and beauty style information are also collected by the device.

[0216] Sending information

[0217] The collected basic information, face photo, and desired beauty style information are compiled in JSON format and sent to the server, which then receives the user's information and begins analyzing it.

[0218] AI-powered analysis and recommendation generation

[0219] The server analyzes the received user information using an artificial intelligence algorithm. The algorithm used here includes, for example, a generative AI model using deep learning. Based on this analysis, the server generates optimal fashion, makeup, and hairstyle suggestions for the user.

[0220] View Suggestions

[0221] The generated suggestions are sent from the server to the user's device, where they are displayed visually. The user can review the suggestions and, if they like them, click the button to display more information.

[0222] Providing supplier information

[0223] Information on where to purchase the suggested fashion and beauty items and beauty salons is also displayed on the device. Users can review this information and click on the purchase or reservation link to be redirected to the purchase website or salon's reservation system.

[0224] Real-time proposals in physical stores

[0225] In-store use allows users to use smart glasses or smartphones to receive real-time fashion and beauty recommendations, allowing them to try on suggested styles in the fitting room or cosmetics section.

[0226] Hardware and software used

[0227] The hardware used includes smartphones, smart glasses, cameras, etc. The software includes Python, OpenCV, and the requests library, which are combined to input, send, analyze, generate suggestions, and display information.

[0228] Specific examples

[0229] For example, suppose a 35-year-old female user enters her age, gender, height 160cm, weight 55kg, skin color light beige, and budget 20,000 yen. A photo of her face is captured, and her desired style of natural makeup and bob cut is selected and sent to the server. The server analyzes the data and suggests a style of autumn casual fashion, natural makeup, and bob cut. These suggestions are displayed in real time on the user's smart glasses, and the user can try them out in the store.

[0230] Prompt Sentence Examples

[0231] "Enter basic information such as your age, gender, height, weight, skin tone, and budget. Then, capture a photo of your face and select the makeup and hairstyle style you want. The AI ​​will then suggest the best fashion, makeup, and hairstyle for you."

[0232] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0233] Step 1: Enter your user information

[0234] Users use a smartphone or smart glasses to input basic information such as age, gender, height, weight, skin color, and budget. The input information is collected by an application installed on the device. This basic information becomes input data. The input data is packaged in JSON format and sent to the server.

[0235] Step 2: Register your face photo and beauty style

[0236] The user captures a photo of their face using the device's camera and inputs the desired makeup and hairstyle. For example, they can select a style such as natural makeup or a bob cut. This data is also compiled in JSON format and sent to the server. In this step, the face photo and beauty style information become input data and are sent to the server.

[0237] Step 3: Analyze the information and generate recommendations

[0238] The server uses an artificial intelligence algorithm to analyze the received data based on the basic information, face photo, and beauty style information. A generative AI model is used in this process. Specifically, based on the input data, the system extracts data features and performs statistical inference to suggest the most suitable fashion and beauty styles for users of the same age. The output is a recommendation of the most suitable fashion, makeup, and hairstyle for the user.

[0239] Step 4: Submit your proposal

[0240] The server generates and sends the proposed results to the user's device. The proposed results consist of both visual (images and graphics) and text formats. This output data is sent to the user's device and received by the application.

[0241] Step 5: Viewing the Suggestion Results

[0242] The user's device displays the received proposal results on the application's user interface. The user can check the displayed proposals and click a button to display detailed information. The input data for this step are the proposal results sent from the server, and the output data is the content displayed on the user interface.

[0243] Step 6: Provide supplier and beauty salon information

[0244] The user's device displays the suggested fashion and beauty item supplier information and beauty salon information to the user. The user can confirm this information and click on the purchase or reservation link to be redirected to the supplier website or salon reservation system. The input data is the supplier information and beauty salon information included in the recommendation results, and the output data is the redirected web page.

[0245] Step 7: Real-time in-store recommendations

[0246] When a user uses smart glasses or a smartphone in a physical store, the device displays appropriate fashion and beauty suggestions in real time. The user can try on the suggested styles in the store. Input data includes the user's current location in the store and a real-time facial photo, and the output data is the suggestions that are best suited to the current situation.

[0247] Step 8: Recording economic behavior data

[0248] The server records the user's purchasing and salon reservation behavior and stores it in an economic behavior database. This data is used to analyze the user's preferences and behavioral patterns. The input data are the purchasing and reservation actions, and the output data is the updated economic behavior database.

[0249] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0250] The present invention relates to a system that combines a user's basic information, facial photo, desired beauty style information, and an emotion engine that recognizes the user's emotions. This makes it possible to provide optimal fashion, makeup, and hairstyle suggestions while taking the user's emotions into consideration. The specific configuration and operation of the system are described below.

[0251] System configuration and operation

[0252] 1. Enter and submit user information

[0253] The user launches the application and enters basic information such as age, gender, height, weight, skin color, budget, etc. The device packages the entered basic information in JSON format and sends it to the server.

[0254] 2. Register your face photo and enter your beauty style

[0255] The user then uploads a photo of their face and selects the desired makeup and hairstyle, which is also compiled into JSON format by the device and sent to the server.

[0256] 3. Emotion Recognition by Emotion Engine

[0257] The server analyzes the received facial photos and uses an emotion engine to recognize the user's emotions, which are then used to adjust fashion and beauty suggestions.

[0258] 4. Comprehensive production by AI

[0259] The server analyzes the received basic information, emotional information, facial photo, and desired beauty style information, and uses an artificial intelligence algorithm to comprehensively produce the most suitable fashion, makeup, and hairstyle for the user. This information is generated as an integrated visual proposal, which is adjusted based on the emotional information. The proposal results are sent from the server to the device.

[0260] 5. Confirmation of proposal and provision of further information

[0261] The device analyzes the recommendations received from the server and displays visual suggestions on the app's interface. The user can review the suggestions and, if they like them, click the "View More Information" button. This will display information about where to purchase the suggested fashion and beauty items and beauty salons.

[0262] 6. Purchase and reservation assistance

[0263] If the user wishes to purchase the suggested item or make a salon reservation, they operate the device and click on the link, which redirects the user to the purchase website or salon reservation system. The server records this action and stores it in the economic behavior database.

[0264] 7. Providing advice on skin type and hair problems

[0265] Additionally, users can input their skin type and hair concerns. The device sends this information to the server, which uses AI to generate appropriate advice and a list of related products. Emotional information is taken into account to provide tailored advice. The device then displays the received advice and product list to the user.

[0266] Specific examples

[0267] Example 1: Entering basic information and beauty information

[0268] A 35-year-old female user logs into the app and enters her age, gender, height 160cm, weight 55kg, skin color light beige, and budget of 20,000 yen. Next, she uploads a photo of her face and selects her desired style of natural makeup and bob cut. The device compiles this information into JSON format and sends it to the server.

[0269] Example 2: Emotion Recognition with an Emotion Engine

[0270] The server analyzes the received facial photo, and the emotion engine recognizes the user's emotion as "happy." This emotion information is used to tailor the suggestions.

[0271] Example 3: Proposal generation and display

[0272] Based on the input basic information, desired style information, and emotional information, the server generates style suggestions, such as autumn casual fashion, natural makeup, and bob cuts, suitable for women of the same age. The results of these suggestions are sent to the device as visuals. The device displays the suggestions to the user, who can click a button to view more information.

[0273] Example 4: Purchasing and reservation assistance

[0274] The user likes the suggested items, a blouse and a skirt, and clicks on a link to purchase them. The device redirects the user to the website where the items are purchased, and the server records this action.

[0275] In this way, the system based on the present invention provides users with customized fashion and beauty suggestions that take emotional information into consideration, creating an environment where they can make purchases and reservations quickly.

[0276] The processing flow will be explained below.

[0277] Program processing steps

[0278] Step 1: Enter your user information

[0279] Behavior 1.1

[0280] Users launch the app and enter basic information such as age, gender, height, weight, skin color, and budget into a form within the app.

[0281] Behavior 1.2

[0282] The device packages the basic information entered in JSON format and sends it to the server.

[0283] Step 2: Register your face photo and enter your beauty style

[0284] Behavior 2.1

[0285] Users can upload a photo of their face to the app using their device's camera or by selecting a suitable photo from their gallery.

[0286] Behavior 2.2

[0287] The user selects the desired makeup and hairstyle (e.g., natural makeup, bob cut).

[0288] Behavior 2.3

[0289] The device compiles the facial photo and beauty style information into JSON format and sends it to the server.

[0290] Step 3: Emotion recognition by the emotion engine

[0291] Behavior 3.1

[0292] The server analyzes the received facial photo using an emotion engine to recognize the user's emotions.

[0293] Behavior 3.2

[0294] The server stores the recognized emotion information in a database and prepares it for further processing.

[0295] Step 4: Comprehensive production by AI

[0296] Behavior 4.1

[0297] The server launches an artificial intelligence algorithm to analyze basic information, emotional information, facial photos, and desired beauty style information.

[0298] Behavior 4.2

[0299] The server's artificial intelligence algorithm generates optimal fashion, makeup, and hairstyle suggestions based on the original input information and emotional information.

[0300] Behavior 4.3

[0301] The server packages the generated proposal results in JSON format and sends them to the terminal.

[0302] Step 5: Review your proposal and provide further information

[0303] Operation 5.1

[0304] The device analyzes the suggestions received from the server and displays visual suggestions on the app interface.

[0305] Behavior 5.2

[0306] The user reviews the suggestions and, if they like them, clicks the button to display more information.

[0307] Operation 5.3

[0308] The device displays a list of suggested fashion and beauty items, as well as information on where to buy them and beauty salons.

[0309] Step 6: Purchase and reservation assistance

[0310] Operation 6.1

[0311] If the user wishes to purchase the suggested item or make a salon appointment, they click on the displayed link.

[0312] Operation 6.2

[0313] Based on the link clicked, the device redirects the user to the purchase website or beauty salon reservation system.

[0314] Operation 6.3

[0315] The server records the user's purchasing and reservation behavior and stores it in an economic behavior database.

[0316] Step 7: Providing advice on skin type and hair concerns

[0317] Operation 7.1

[0318] Users can also enter their skin type and hair concerns (e.g., dry skin, dandruff, etc.) into the app.

[0319] Operation 7.2

[0320] The device packages the entered information about skin type and hair concerns in JSON format and sends it to the server.

[0321] Operation 7.3

[0322] The server uses artificial intelligence algorithms to generate appropriate advice and a list of related products based on the input skin type and hair concerns, while also taking emotional information into account.

[0323] Operation 7.4

[0324] The server packages the generated advice and product list in JSON format and sends it to the terminal.

[0325] Operation 7.5

[0326] The device displays the received advice and a list of related products on the app interface.

[0327] Specific examples

[0328] Example 1: Entering basic information and beauty information

[0329] A 35-year-old female user logs into the app and enters her age, gender, height 160cm, weight 55kg, skin color light beige, and budget of 20,000 yen. Next, she uploads a photo of her face and selects her desired style of natural makeup and bob cut. The device compiles this information into JSON format and sends it to the server.

[0330] Example 2: Emotion Recognition with an Emotion Engine

[0331] The server analyzes the received facial photo, and the emotion engine recognizes the user's emotion as "happy." This emotion information is used to tailor the suggestions.

[0332] Example 3: Proposal generation and display

[0333] Based on the input basic information, desired style information, and emotional information, the server generates style suggestions, such as autumn casual fashion, natural makeup, and bob cuts, suitable for women of the same age. The results of these suggestions are sent to the device as visuals. The device displays the suggestions to the user, who can click a button to view more information.

[0334] Example 4: Purchasing and reservation assistance

[0335] The user likes the suggested items, a blouse and a skirt, and clicks on a link to purchase them. The device redirects the user to the website where the items are purchased, and the server records this action.

[0336] By following these steps, the system based on the present invention provides users with customized fashion and beauty suggestions that take emotional information into account, creating an environment for quick purchases and reservations.

[0337] Example 2

[0338] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0339] Current beauty and fashion recommendation systems often do not adequately consider users' emotions or detailed personal information. As a result, the recommendations provided may not meet users' expectations. Furthermore, many systems do not include a means to smoothly complete purchase or reservation procedures, resulting in a lack of convenience for users. This poses a challenge, resulting in a decrease in user satisfaction.

[0340] The identification process by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes: means for inputting basic information about the user; means for inputting a facial photo and desired beauty style information about the user; artificial intelligence algorithm means for analyzing the input information and making optimal fashion, makeup, and hairstyle suggestions; emotion recognition means for analyzing the received facial photo and recognizing the user's emotions; means for generating suggestions and adjusting them based on the emotion information; means for providing information about where to purchase the suggested fashion and beauty items and beauty salons; and means for providing appropriate advice and a list of related products based on the user's skin type or hair concerns. This makes it possible to provide more personalized suggestions that take into account the user's emotions and detailed personal information, facilitating smoother purchasing and reservation procedures.

[0341] "Basic information" refers to personal information such as the user's age, gender, height, weight, skin color, and budget.

[0342] "Face Photo" refers to an image of a user's face uploaded by the user.

[0343] "Beauty style information" refers to information about the user's desired beauty style, such as desired makeup and hairstyle.

[0344] "Artificial intelligence algorithm" refers to an algorithm that analyzes input information and generates optimal fashion suggestions, makeup suggestions, and hairstyle suggestions.

[0345] "Emotion recognition means" refers to a system that analyzes the received facial photo and recognizes the user's emotions.

[0346] The "means for adjusting the recommendation result" refers to a means for adjusting the generated recommendation result based on the recognized emotion.

[0347] "Purchaser Information" refers to information about where to purchase suggested fashion and beauty items.

[0348] "Beauty salon information" refers to information about a beauty salon that provides the proposed beauty services.

[0349] "Skin type" refers to the nature and condition of the user's skin.

[0350] "Hair concerns" refers to the hair-related problems and hopes that users have.

[0351] "Advice means" refers to a means for providing appropriate advice based on the user's skin type and hair concerns.

[0352] "Related Product Listings" refers to lists of beauty products or items recommended based on the advice provided.

[0353] "Means for inputting basic information" refers to the interface and functions that allow users to input basic information.

[0354] "Means for inputting face photo and beauty style information" refers to an interface or function that allows a user to input a face photo and desired beauty style information.

[0355] The present invention relates to a system that integrates a user's basic information, facial photo, desired beauty style information, and emotional information to provide optimal fashion, makeup, and hairstyle suggestions. This system can personalize suggestions using an emotion engine that recognizes the user's emotions and an artificial intelligence algorithm. The specific configuration and operation of the present invention are described below.

[0356] composition

[0357] 1. Enter and submit user information

[0358] The user launches the application and enters basic information such as age, gender, height, weight, skin color, and budget. The device packages the entered basic information in JSON format and sends it to the server. The software used is an interface for entering basic information and a program for converting it to JSON format.

[0359] 2. Register your face photo and enter your beauty style

[0360] The user then uploads a photo of their face and selects the desired makeup and hairstyle. This is also compiled into JSON format by the device and sent to the server. The software used is an interface for uploading face photos and selecting beauty styles, as well as a comprehensive data package program.

[0361] 3. Emotion Recognition by Emotion Engine

[0362] The server analyzes the received facial photos and uses an emotion engine to recognize the user's emotions. This emotion information is used to adjust fashion and beauty suggestions. The hardware used is a high-performance image analysis server, and the software is an emotion recognition engine (e.g., Google® Cloud Vision API).

[0363] 4. Comprehensive production by AI

[0364] The server analyzes the received basic information, emotional information, facial photo, and desired beauty style information, and uses artificial intelligence algorithms to comprehensively produce the most suitable fashion, makeup, and hairstyle for the user. This information is generated as an integrated visual proposal, which is adjusted based on the emotional information. The proposal results are sent from the server to the device. The software used is a machine learning model (e.g., TensorFlow) and an image generation algorithm.

[0365] 5. Confirmation of proposal and provision of further information

[0366] The device analyzes the recommendation results received from the server and displays visual suggestions on the app's interface. The user checks the suggestions and clicks the display button if more information is needed. This displays information on where to purchase the suggested fashion and beauty items and beauty salons. The software used is the recommendation result display interface and the detailed information display program.

[0367] 6. Purchase and reservation assistance

[0368] If the user wishes to purchase the suggested item or make a salon reservation, they operate the device and click on the link. This causes the device to redirect the user to the purchasing website or salon reservation system. The server records this behavior and stores it in an economic behavior database. The software used is a redirect program and a behavior recording program.

[0369] 7. Providing advice on skin type and hair problems

[0370] Additionally, users can enter their skin type and hair concerns. The device sends this information to a server, which uses AI to generate appropriate advice and a list of related products. Emotional information is taken into account to provide tailored advice. The device then displays the received advice and product list to the user. The software used is a skin type and hair concern input interface and a program that generates a list of related products.

[0371] Specific examples

[0372] Example 1: Entering basic information and beauty information

[0373] A 35-year-old female user logs into the app and enters her age, gender, height 160cm, weight 55kg, skin color light beige, and budget of 20,000 yen. Next, she uploads a photo of her face and selects her desired style of natural makeup and bob cut. The device compiles this information into JSON format and sends it to the server.

[0374] Example 2: Emotion Recognition with an Emotion Engine

[0375] The server analyzes the received facial photo, and the emotion engine recognizes the user's emotion as "happy." This emotion information is used to tailor the suggestions.

[0376] Example 3: Proposal generation and display

[0377] Based on the user's basic information, desired style, and emotional information, the server proposes styles suitable for women of the same age, such as autumn casual fashion, natural makeup, and bob cuts. The proposed results are sent to the device as visuals. The device displays the proposed content to the user, who can click a button to view more information.

[0378] Example 4: Purchasing and reservation assistance

[0379] The user likes the suggested items, a blouse and a skirt, and clicks on a link to purchase them. The device redirects the user to the website where the items are purchased, and the server records this action.

[0380] Prompt Sentence Examples

[0381] "Please suggest something that suits the casual fashion and natural makeup worn by a 35-year-old woman. Furthermore, the user is showing the emotion 'happy.'"

[0382] This allows users to get the best fashion and beauty suggestions from the generative AI model.

[0383] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0384] Step 1:

[0385] A user launches the application and logs in. They then enter basic information such as age, gender, height, weight, skin tone, and budget.

[0386] Input: Basic information entered by the user.

[0387] Specific operation: User authentication is performed on the application login screen, and after moving to the main screen, a basic information input form is displayed. When the user clicks the submit button, the input data is converted to JSON format on the terminal side.

[0388] Output: Basic information converted to JSON format.

[0389] Step 2:

[0390] The terminal transmits the converted basic information to the server.

[0391] Input: Basic information in JSON format.

[0392] Specific operation: The device sends JSON data to the server using the REST API.

[0393] Output: Basic information data received by the server.

[0394] Step 3:

[0395] Users upload a photo of their face and select the desired makeup and hairstyle.

[0396] Input: A photo of the user's face and desired beauty style.

[0397] Specific operation: A face photo upload screen is displayed, the user selects a photo, clicks the upload button, and after selecting a beauty style, the device compiles this information into JSON format.

[0398] Output: A JSON-formatted face photo and desired beauty style information.

[0399] Step 4:

[0400] The device sends a facial photo and beauty style information in JSON format to the server.

[0401] Input: Face photo and beauty style information in JSON format.

[0402] Specific operation: The device sends data to the server using the REST API.

[0403] Output: Face photo and beauty style information data received by the server.

[0404] Step 5:

[0405] The server analyzes the received facial photograph and recognizes the user's emotions using an emotion recognition engine.

[0406] Input: Facial photo data.

[0407] Specific operation: The server uses a high-performance image analysis server to run a facial recognition algorithm and analyzes emotions using an emotion recognition engine (e.g., Google Cloud Vision API).

[0408] Output: Emotion information data.

[0409] Step 6:

[0410] The server analyzes basic information, emotional information, facial photos, and desired beauty style information, and uses artificial intelligence algorithms to produce the optimal fashion, makeup, and hairstyle.

[0411] Input: Basic information, emotional information, face photo, desired beauty style information.

[0412] How it works: The server analyzes this information using a data analysis module and generates suggestions using a machine learning model (e.g., TensorFlow), which are then visualized using an image generation algorithm.

[0413] Output: Visual suggestion data.

[0414] Step 7:

[0415] The server transmits the visual suggestion data to the terminal.

[0416] Input: Visual suggestion data.

[0417] Specific operation: The server sends visual suggestion data to the device using the REST API.

[0418] Output: Visual suggestion data received by the device.

[0419] Step 8:

[0420] The device analyzes the received suggestions and displays visual suggestions on the app interface. The user can review the suggestions and click the view button to view more information.

[0421] Input: Visual suggestion data.

[0422] What happens: The device parses the received content and displays it in the app's UI as images and text. When the user clicks the details button, the additional information expands to show more information.

[0423] Output: Visual suggestions and detailed information displayed in the app's UI.

[0424] Step 9:

[0425] If the user wishes to purchase the suggested item or make a salon reservation, he or she operates the terminal and clicks on the link.

[0426] Input: Purchase or reservation link.

[0427] What happens: When a user clicks on a purchase link, their device's browser opens to the website where the purchase is made.

[0428] Output: Purchasing website or salon booking system.

[0429] Step 10:

[0430] The server records user behavior and stores it in an economic behavior database.

[0431] Input: User behavior data.

[0432] Specific operation: The server logs users' link clicks and purchase behavior and stores them in an economic behavior database.

[0433] Output: Stored behavioral data.

[0434] Step 11:

[0435] The user enters additional information such as their skin type and hair concerns, and the device sends this to the server.

[0436] Input: Skin type and hair concerns.

[0437] Specific behavior: When the user fills out the additional information input form and clicks the submit button, the device converts the data into JSON format and sends it to the server.

[0438] Output: Skin type and hair problem information received by the server.

[0439] Step 12:

[0440] The server uses AI to generate appropriate advice and a list of related products based on skin type and hair concerns.

[0441] Input: Skin type and hair concerns.

[0442] How it works: The server uses an AI model to analyze the input information and generate advice and product lists.

[0443] Output: Advice and related product list.

[0444] Step 13:

[0445] The terminal displays the received advice and related product list to the user.

[0446] Input: Advice and related product listings.

[0447] Specific behavior: The device parses the received content and displays it in the app's UI.

[0448] Output: Advice and related product list displayed in the app UI.

[0449] (Application example 2)

[0450] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0451] Today's consumers demand personalized fashion and beauty recommendations, but existing systems often only offer one-size-fits-all recommendations, resulting in low satisfaction. Furthermore, the effectiveness of recommendations is limited because they are not tailored based on real-time emotional information. Furthermore, there is a lack of support for in-store purchases and try-ons, creating a need for ways to improve the user shopping experience.

[0452] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes: means for inputting basic information about the user; means for inputting a facial photo and desired beauty style information about the user; means including an artificial intelligence algorithm for analyzing the input information and making optimal fashion, makeup, and hairstyle suggestions; means for providing information on where to purchase the suggested fashion and beauty items and beauty salon information; means for providing appropriate advice and a list of related products based on the user's skin type or hair concerns; means for displaying the suggestion results on smart glasses worn by the user in a physical store, recognizing emotions in real time, and adjusting the suggestion content based on the emotional information; and means for supporting the user in trying on and purchasing items they like through the smart glasses. This enables the provision of personalized suggestions and adjustments based on real-time emotional information, as well as an improved shopping experience in a physical store.

[0453] "Means for inputting basic user information" refers to devices or applications for inputting basic information such as the user's age, gender, height, weight, skin color, and budget.

[0454] The "means for inputting the user's facial photograph and desired beauty style information" refers to a device or application for inputting the user's facial photograph and beauty style information such as desired makeup and hairstyle.

[0455] "Means including an artificial intelligence algorithm for analyzing input information and making optimal suggestions for fashion, makeup, and hairstyle" refers to systems or software that include an artificial intelligence algorithm used to analyze input user information and make suggestions for optimal fashion, makeup, and hairstyle.

[0456] The "means for providing information on where to purchase suggested fashion and beauty items and information on beauty salons" refers to a system or application for providing information on where to purchase suggested fashion and beauty items and information on beauty salons.

[0457] "Means for providing appropriate advice and a list of related products based on the user's skin type or hair concerns" refers to a system or software for providing appropriate advice and a list of related products based on the user's skin type or hair concerns.

[0458] "A means for displaying the proposal results on smart glasses worn by the user in a physical store, recognizing emotions in real time, and adjusting the proposal content based on that emotional information" refers to a system or algorithm for displaying the proposal results on smart glasses worn by the user in a physical store, recognizing the user's emotions in real time, and adjusting the proposal content based on that emotional information.

[0459] "Means for supporting users in trying on and purchasing items they like through smart glasses" refers to a system or software that supports users in trying on and purchasing items they like through the UI of smart glasses.

[0460] The present invention relates to a system that combines a user's basic information, facial photo, desired beauty style information, and an emotion engine that recognizes the user's emotions. This makes it possible to provide optimal fashion, makeup, and hairstyle suggestions while taking the user's emotions into consideration. The specific configuration and operation of the system are described below.

[0461] System configuration and operation

[0462] 1. Enter and submit user information

[0463] Users wear smart glasses and input basic information such as age, gender, height, weight, skin color, and budget. The smart glasses package the input basic information in JSON format and send it to the server. The smart glasses are equipped with a camera and touchpad, making it easy for users to input information.

[0464] 2. Register your face photo and enter your beauty style

[0465] The user then takes a photo of their face using the smart glasses' camera and selects the desired makeup and hairstyle, which is then compiled into JSON format by the smart glasses and sent to the server.

[0466] 3. Emotion Recognition by Emotion Engine

[0467] The server analyzes the received facial photo and uses an emotion engine (emotion recognition technology such as DeepFace) to recognize the user's emotions. This emotional information is used to adjust fashion and beauty suggestions.

[0468] 4. Comprehensive production by AI

[0469] The server analyzes the received basic information, emotional information, facial photo, and desired beauty style information, and then uses an artificial intelligence algorithm (e.g., a model using Keras) to comprehensively produce the most suitable fashion, makeup, and hairstyle for the user. This information is then generated as an integrated visual suggestion, which is adjusted based on the emotional information. The suggestion results are sent from the server to the smart glasses.

[0470] 5. Confirmation of proposal and provision of further information

[0471] The smart glasses analyze the recommendations received from the server and display visual suggestions to the user. The user reviews the recommendations and, if they like them, clicks the display details button on the touchpad. This displays information on where to purchase the suggested fashion and beauty items and beauty salons on the smart glasses' display.

[0472] 6. Purchase and reservation assistance

[0473] If the user chooses a suggested item and wishes to purchase or try it on, they can operate the smart glasses' interface and click the corresponding button, which will redirect the user to the purchase website or salon's reservation system. The server records this action and stores it in the economic behavior database.

[0474] 7. Providing advice on skin type and hair problems

[0475] Additionally, users can input their skin type and hair concerns through the smart glasses. The smart glasses then send this information to the server, which uses AI to generate appropriate advice and a list of related products. Emotional information is taken into account to provide tailored advice. The smart glasses then display the received advice and product list to the user.

[0476] Specific examples

[0477] Example 1: Entering basic information and beauty information

[0478] A 30-year-old female user puts on smart glasses and enters her age, gender, height 160cm, weight 55kg, skin color light beige, and budget of 20,000 yen. Next, she takes a photo of her face with the smart glasses' camera and selects her desired style of natural makeup and short hair. The smart glasses then compile this information into JSON format and send it to the server.

[0479] Example 2: Emotion Recognition with an Emotion Engine

[0480] The server analyzes the received facial photo and uses an emotion engine (DeepFace) to recognize the user's emotion as "happy." This emotion information is used to adjust the suggestions.

[0481] Example 3: Proposal generation and display

[0482] Based on the input basic information, desired style information, and emotional information, the server generates style suggestions, such as autumn casual fashion, natural makeup, and short haircuts, suitable for women of the same age. The results of these suggestions are sent as visuals to the smart glasses. The smart glasses then display the suggestions to the user, who can then operate the touchpad to view more information.

[0483] Example 4: Purchasing and reservation assistance

[0484] The user likes the suggested items, a jacket and pants, and clicks on a link to purchase them. The smart glasses redirect the user to the website where they purchased the items, and the server records this action.

[0485] Prompt Sentence Examples

[0486] Recognize user emotions and provide customized fashion and beauty recommendations based on the following information:

[0487] Age: 30

[0488] Gender: Female

[0489] Height: 160cm

[0490] Weight: 55kg

[0491] Skin Tone: Light Beige

[0492] Budget: 20,000 yen

[0493] Face photo: face.jpg

[0494] Suggest the best style based on user sentiment.

[0495] In this way, the system based on the present invention provides users with customized fashion and beauty suggestions that take emotional information into account, allowing for real-time adjustments and quick purchase and try-on in-store.

[0496] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0497] Step 1:

[0498] The user puts on the smart glasses and inputs basic information, such as age, gender, height, weight, skin tone, and budget, using the touchpad on the smart glasses interface. The input basic information is converted into JSON format and sent to the server, where the user information is stored and available for subsequent analysis.

[0499] Step 2:

[0500] The user takes a photo of their face using the smart glasses' camera and inputs their desired beauty style information (e.g., type of makeup, hairstyle). The smart glasses also compile this information into JSON format and send it to the server. The server then receives the user's face photo and desired style information, and is ready for emotion recognition.

[0501] Step 3:

[0502] The server analyzes the received facial photo and recognizes the user's emotions using an emotion engine (such as DeepFace). Specifically, it uses the DeepFace library to extract facial features from the facial photo and determine the emotion based on those features. The emotional information obtained here is then used to adjust the fashion suggestions.

[0503] Step 4:

[0504] The server combines the user's basic information, face photo, desired beauty style information, and emotional information received from the user and analyzes it using an artificial intelligence algorithm (e.g., a generative AI model using Keras). This analysis generates optimal fashion, makeup, and hairstyle recommendations. The AI ​​converts the user's information into personalized recommendations based on statistical data and trend information.

[0505] Step 5:

[0506] The server sends the generated suggestions to the smart glasses worn by the user in the physical store. The smart glasses display the suggestions to the user. The user can review the suggestions using the touchpad, and if they like them, they can access more detailed information by tapping the display button for more information.

[0507] Step 6:

[0508] If the user likes any of the suggested fashion items, they will be assisted in trying them on and purchasing them through the smart glasses' UI. Specifically, the smart glasses will provide fitting room reservations and detailed product information based on the user's preferences, and guide the user through the operations required for trying on and purchasing. This allows the user to quickly try on and purchase items using the smart glasses.

[0509] Step 7:

[0510] The server records the user's economic behavior data and stores it in a database. Specifically, it collects and records behavioral data such as which offers the user accepted, which items they purchased, and which beauty salons they booked. This data can be used for subsequent analysis and to improve the accuracy of future offers.

[0511] Step 8:

[0512] Users also input their skin type and hair concerns through the smart glasses. The smart glasses send this information to a server, which uses AI to generate appropriate advice and a list of related products. The advice takes emotional information into account, and is received by the smart glasses and displayed to the user.

[0513] Through these steps, users can receive personalized fashion and beauty recommendations based on their emotions, with real-time adjustments and a quick in-store purchasing experience.

[0514] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0515] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (registered trademark) (Internet search engine).<URL: https: / / openai.com / blog / chatgpt> ), Gemini (registered trademark) (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0516] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0517] [Second embodiment]

[0518] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0519] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0520] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0521] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0522] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0523] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0524] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0525] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0526] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0527] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0528] In the smart glasses 214, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0529] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0530] The present invention relates to a system that utilizes artificial intelligence to provide a means for users to input their basic information, a photograph of their face, and information about their desired beauty style, and then analyzes this information to provide comprehensive fashion, makeup, and hairstyle suggestions. This system automatically provides optimal fashion and beauty suggestions by having users input the necessary information.

[0531] System configuration and operation

[0532] 1. Enter and submit user information

[0533] The user launches the application and enters basic information such as age, gender, height, weight, skin tone, budget, etc. The device packages the entered information in JSON format and sends it to the server.

[0534] 2. Register your face photo and beauty style

[0535] The user then uploads a photo of their face and selects the desired makeup and hairstyle, which is also compiled into JSON format by the device and sent to the server.

[0536] 3. AI-generated recommendations

[0537] Based on the received information, the server uses an artificial intelligence algorithm to comprehensively produce the most suitable fashion, makeup, and hairstyle for the user. This information is analyzed and an integrated visual proposal is generated. The proposal results are then sent from the server to the device.

[0538] 4. Review the proposal and view more information

[0539] The device receives the recommendations from the server and displays them to the user on the app interface. The user can review the recommendations and, if they like them, click the "Display details" button. This will display information about where to purchase the recommended fashion and beauty items and beauty salons.

[0540] 5. Purchase and reservation assistance

[0541] If the user wishes to purchase the suggested item or make a salon reservation, they operate the device and click on the link, which redirects the user to the purchase website or salon reservation system. The server records this action and stores it in the economic behavior database.

[0542] 6. Providing advice on skin type and hair problems

[0543] Additionally, users can input their skin type and hair concerns. The device sends this information to the server, which uses AI to generate appropriate advice and a list of related products and send them to the device. The device then displays the received advice and product list to the user.

[0544] Specific examples

[0545] Example 1: Entering basic information and beauty information

[0546] A 35-year-old female user logs into the app and enters her age, gender, height 160cm, weight 55kg, skin color light beige, and budget of 20,000 yen. Next, she uploads a photo of her face and selects her desired style of natural makeup and bob cut. The device compiles this information into JSON format and sends it to the server.

[0547] Example 2: Proposal generation and display

[0548] The server analyzes the input information and generates style suggestions, such as autumn casual fashion, natural makeup, and bob cuts, suitable for women of the same age. These suggestions are sent as visuals to the device. The device displays the suggestions to the user, who can click a button to view more information.

[0549] Example 3: Purchasing and reservation assistance

[0550] The user likes the suggested items, a blouse and a skirt, and clicks on a link to purchase them. The device redirects the user to the website where the items are purchased, and the server records this action.

[0551] In this way, the system based on the present invention provides users with quick and accurate comprehensive fashion and beauty suggestions, and smoothly links these to purchases and service use.

[0552] The processing flow will be explained below.

[0553] Program processing steps

[0554] Step 1: Enter your user information

[0555] Behavior 1.1

[0556] Users launch the app and enter basic information such as age, gender, height, weight, skin color, and budget into a form within the app.

[0557] Behavior 1.2

[0558] The device packages the basic information entered in JSON format and sends it to the server.

[0559] Step 2: Register your face photo and enter your beauty style

[0560] Behavior 2.1

[0561] Users can upload a photo of their face to the app using their device's camera or by selecting a suitable photo from their gallery.

[0562] Behavior 2.2

[0563] The user selects the desired makeup and hairstyle (e.g., natural makeup, bob cut).

[0564] Behavior 2.3

[0565] The device compiles the facial photo and beauty style information into JSON format and sends it to the server.

[0566] Step 3: Comprehensive production by AI

[0567] Behavior 3.1

[0568] The server then calls an artificial intelligence algorithm to analyze the received basic information, facial photo, and desired beauty style information.

[0569] Behavior 3.2

[0570] The server's artificial intelligence algorithm generates optimal fashion, makeup, and hairstyle suggestions based on the user's age, gender, height, weight, skin color, budget, and desired style information.

[0571] Behavior 3.3

[0572] The server packages the generated proposal results in JSON format and sends them to the terminal.

[0573] Step 4: Review your proposal and provide further information

[0574] Behavior 4.1

[0575] The device analyzes the suggestions received from the server and displays visual suggestions on the app interface.

[0576] Behavior 4.2

[0577] The user reviews the suggestions and, if they like them, clicks the button to display more information.

[0578] Behavior 4.3

[0579] The device displays a list of suggested fashion and beauty items, as well as information on where to purchase them and beauty salons.

[0580] Step 5: Purchase and reservation assistance

[0581] Operation 5.1

[0582] If the user wishes to purchase the suggested item or make a salon appointment, they click on the displayed link.

[0583] Behavior 5.2

[0584] Based on the link clicked, the device redirects the user to the purchase website or beauty salon reservation system.

[0585] Operation 5.3

[0586] The server records the user's purchasing and reservation behavior and stores it in an economic behavior database.

[0587] Step 6: Providing advice on skin type and hair concerns

[0588] Operation 6.1

[0589] Users can also enter their skin type and hair concerns (e.g., dry skin, dandruff, etc.) into the app.

[0590] Operation 6.2

[0591] The device packages the entered information about skin type and hair concerns in JSON format and sends it to the server.

[0592] Operation 6.3

[0593] The server uses artificial intelligence algorithms to generate appropriate advice and a list of related products based on the input skin type and hair concerns.

[0594] Operation 6.4

[0595] The server packages the generated advice and product list in JSON format and sends it to the terminal.

[0596] Operation 6.5

[0597] The device displays the received advice and a list of related products on the app interface.

[0598] ---

[0599] Through these steps, users can receive customized fashion and beauty suggestions through a consistent process, and can quickly make purchases or reservations.

[0600] Example 1

[0601] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0602] Conventional beauty and fashion recommendation systems have the problem that it takes a lot of time and effort for users to match their own information and preferences in detail. In addition, there are few systems that integrate various information to make comprehensive recommendations, and the purchasing and reservation process after the recommendations is often fragmented. This has led to issues such as low user satisfaction and a lack of improvement in system usage efficiency.

[0603] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0604] In this invention, the server includes means for inputting the user's personal information, means for inputting the user's face photo and desired beauty style information, means for packaging the input information into a data format and sending it to the server, means for analyzing the input information using an artificial intelligence algorithm and making optimal fashion, makeup, and hairstyle suggestions, means for displaying the suggestions received from the server to the user, means for providing information on where to purchase the suggested fashion and beauty items and information on beauty salons, and means for providing appropriate advice and a list of related products based on the user's skin type or hair concerns. This allows users to input their own information within a single system, receive optimal fashion and beauty suggestions based on that information, and also enable them to purchase related items and make beauty salon reservations in an integrated manner.

[0605] "User" means an individual who uses the System.

[0606] "Personal information" refers to information such as age, gender, height, weight, skin color, and budget that users enter into the system.

[0607] "Facial photo" refers to image data of a user's face that the user uploads to the system.

[0608] "Beauty style information" refers to information about the makeup and hairstyle style desired by the user.

[0609] A "data format" refers to a format that structures information to make it easier to store and communicate; a commonly used example is the JSON format.

[0610] "Server" refers to the central computer of the system, a computing resource that receives user input, analyzes it using artificial intelligence algorithms, and makes suggestions.

[0611] "Artificial intelligence algorithm" refers to machine learning and deep learning models and methods used to analyze input data and generate optimal fashion, makeup, and hairstyle suggestions.

[0612] "Suggestion results" is a general term for fashion suggestions, makeup suggestions, and hairstyle suggestions that are analyzed by an artificial intelligence algorithm and provided to users.

[0613] "Purchaser Information" refers to information about stores and online stores where suggested fashion and beauty items can be purchased.

[0614] "Beauty salon information" refers to information such as the location and contact details of the beauty salon that offers the proposed beauty style.

[0615] "Skin type" refers to the condition or characteristics of the user's skin (e.g., dry skin, oily skin).

[0616] "Hair concerns" refers to problems with the user's hair or areas they would like to improve (e.g., thinning hair, split ends).

[0617] The "List of Related Products" refers to a list of beauty products recommended based on the user's skin type and hair concerns.

[0618] "Purchase Intent" refers to a state in which a user has indicated an intention to purchase or use a proposed item or service.

[0619] "Redirecting" refers to the act of sending a user to another web page or service.

[0620] "Economic behavior data" refers to behavioral data when users engage in economic activities such as purchases and reservations.

[0621] "Recording and storing" refers to keeping specific information in a database or storage so that it can be accessed later.

[0622] The present invention relates to a system that utilizes artificial intelligence to input a user's basic information, a facial photo, and desired beauty style information, and then analyzes the information to provide comprehensive fashion, makeup, and hairstyle suggestions. This system automatically provides optimal fashion and beauty suggestions by having the user input the necessary information. Specific embodiments are described in detail below.

[0623] Enter and submit user information

[0624] Users launch a dedicated application and enter their personal information, including age, gender, height, weight, skin color, and budget. This information is packaged in JSON format by the device and sent to the server. Users also upload a photo of their face and select their desired beauty style (e.g., natural makeup, bob cut). This information is also packaged in JSON format by the device and sent to the server.

[0625] AI-powered recommendation generation

[0626] The server analyzes the received user information using an artificial intelligence algorithm. This algorithm uses machine learning models such as TensorFlow and PyTorch. The server generates optimal fashion, makeup, and hairstyle suggestions for the user, renders them in a visual format, and sends them to the device. The analyzed and generated suggestions are then sent to the device as JSON-formatted data.

[0627] Displaying the proposed results and providing detailed information

[0628] The device visually displays the received suggestions on the dedicated application interface. Users can check the suggestions displayed and click the confirmation button for more information, which will display information on where to purchase the suggested fashion items and detailed information on beauty salons.

[0629] Purchase and reservation assistance

[0630] If the user wishes to purchase the suggested item or make a reservation at a beauty salon, they click on the provided link on their device. By clicking on the link, the device redirects the user to the product purchase site or beauty salon reservation system. The user's clicking behavior is recorded by the server and later stored in the economic behavior database.

[0631] Advice on skin and hair problems

[0632] Additionally, users can enter additional information about their skin type and hair concerns. The device sends this information to the server, which uses AI algorithms to generate a list of appropriate advice and related products. This list and advice is then sent to the device and displayed to the user.

[0633] Examples and prompts

[0634] Example 1: Entering basic information and beauty information

[0635] A 35-year-old female user logs into the app and enters her age, height 160cm, weight 55kg, skin color light beige, and budget of 20,000 yen. Next, she uploads a photo of her face and selects her desired style of natural makeup and bob cut. The device compiles this information into JSON format and sends it to the server.

[0636] Example 2: Proposal generation and display

[0637] The server analyzes the input information and generates recommendations for autumn casual fashion, natural makeup, and a bob cut that are ideal for a 35-year-old woman, and sends them to the device. The device displays the recommendations and provides more information when the user clicks the "More Info" button.

[0638] Prompt Sentence Examples

[0639] "35-year-old female, height 160cm, weight 55kg, skin tone light beige, budget 20,000 yen. Prefer natural makeup and a bob cut. Please provide suggestions for casual fall fashion and specific shopping information."

[0640] The above is a detailed description of the embodiment of the present invention. This system allows users to efficiently and easily receive suggestions for optimal fashion and beauty styles, and then smoothly take the next action, such as purchasing a product or making a reservation at a beauty salon, based on that information.

[0641] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0642] Step 1: Enter and submit user information

[0643] The user starts the application and enters personal information (age, sex, height, weight, skin color, budget, etc.) by typing this information into a dedicated form.

[0644] Input: Personal information entered by the user

[0645] The terminal packages the entered information into JSON format.

[0646] Data processing: Validate the information entered by the user and convert it to JSON format.

[0647] Output: User details in JSON format

[0648] The terminal sends the generated JSON data to the server via an HTTP POST request.

[0649] Step 2: Register your face photo and beauty style

[0650] Users upload a photo of their face and select the desired beauty style (e.g., natural makeup, bob cut). Specifically, they take a photo of their face using the camera function on their smartphone and upload it to the app.

[0651] Input: Uploaded face photo and selected beauty style information

[0652] The device then reassembles this information into JSON format.

[0653] Data processing: Convert the face photo into the appropriate image format (e.g., JPEG) and include beauty style information in the JSON data.

[0654] Output: JSON formatted face photo and beauty style information

[0655] The terminal sends the generated JSON data to the server via an HTTP POST request.

[0656] Step 3: Generate recommendations with AI

[0657] The server analyzes the received JSON-formatted user information, face photo, and beauty style information using an artificial intelligence model (e.g., TensorFlow, PyTorch).

[0658] Input: JSON format user information, face photo, beauty style information

[0659] The server inputs this information into a generative AI model to generate optimal fashion, makeup, and hairstyle suggestions.

[0660] Data calculation: Generative AI models analyze input information and generate optimal recommendations

[0661] Output: JSON format suggestion results (fashion suggestions, makeup suggestions, hairstyle suggestions)

[0662] The server renders the generated proposal results in a visual format and sends them to the terminal as JSON data.

[0663] Step 4: View the results of the proposal and provide more information

[0664] The device displays the received proposal results on the interface of a dedicated app. The proposals are visually easy to understand.

[0665] Input: JSON formatted proposal results

[0666] Output: Visual suggestion results shown to the user

[0667] The user checks the suggested content and clicks the "Display detailed information" button, which displays information about where to purchase the suggested fashion items and detailed information about beauty salons.

[0668] Step 5: Assist with purchases and reservations

[0669] If users want to purchase or book a featured item or beauty service, they click on a link within the app.

[0670] Input: User clicks

[0671] The terminal responds to the click and redirects the user to a purchasing website or a beauty salon reservation system.

[0672] Data calculation: Link click event and redirect handling

[0673] Output: The redirected web page

[0674] The server records the user's clicking behavior and stores it in an economic behavior database.

[0675] Step 6: Providing advice on skin type and hair concerns

[0676] Users can also enter their skin type and hair concerns (e.g., dry skin, thinning hair).

[0677] Input: Information about your skin type and hair concerns

[0678] The device compiles this information into JSON format and sends it to the server.

[0679] The server passes the received information to an AI algorithm to generate optimal advice and related product lists.

[0680] Data arithmetic: AI algorithms analyze information and generate advice

[0681] Output: JSON formatted advice and related product list

[0682] The terminal displays the received information again to the user, allowing the user to check appropriate advice and related products.

[0683] (Application example 1)

[0684] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0685] Conventional fashion and beauty recommendation systems often require users to input information online and wait for a long time before receiving recommendations. They are also difficult to use in physical stores, lacking real-time support for users to try on clothes and check makeup looks. Furthermore, they lack the means to provide personalized advice based on a user's skin type or hair concerns. Therefore, there was a need for a system that provides optimal fashion and beauty recommendations in real time in physical stores and supports users in making their choices.

[0686] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0687] In this invention, the server includes means for inputting basic information about a user, means for inputting a face photo and desired beauty style information about the user, means including an artificial intelligence algorithm for analyzing the input information and making optimal fashion, makeup, and hairstyle suggestions, means for providing information on where to purchase the suggested fashion and beauty items and beauty salon information, means for providing appropriate advice and a list of related products based on the user's skin type or hair concerns, and means for receiving real-time fashion and beauty suggestions in a physical store via a smart device. This allows users to receive optimal fashion and beauty suggestions in real time in a physical store, enabling them to efficiently and effectively choose fashion and decide on a beauty style.

[0688] "Basic information" refers to information about a user's personal characteristics and attributes, such as age, sex, height, weight, skin color, and budget.

[0689] A "face photo" is image data of a photograph of a user's face.

[0690] "Beauty style information" is information about the makeup and hairstyle style desired by the user.

[0691] An "artificial intelligence algorithm" is a calculation method that analyzes input information and automatically generates optimal fashion, makeup, and hairstyle suggestions for users.

[0692] "Purchase Information" refers to information about locations or web pages where suggested fashion and beauty items can be purchased.

[0693] "Beauty salon information" refers to information about the location of stores that offer beauty services and how to make reservations.

[0694] "Skin type" refers to the type and characteristics of the user's skin, such as oily skin, dry skin, or sensitive skin.

[0695] "Hair problems" refers to problems or concerns that users have with their hair, such as hair loss, split ends, and dandruff.

[0696] "Appropriate advice" refers to solutions and care methods provided based on the user's skin type and hair concerns.

[0697] "Related Product Listings" refers to a list of products or services related to the relevant advice.

[0698] "Smart devices" refers to portable information devices that can connect to the Internet, such as smartphones and smart glasses.

[0699] "In real time in the physical store" means that suggestions and information are provided instantly on the spot while the customer is in the store.

[0700] To implement this invention, a system is required in which users, terminals, and servers each play a specific role.

[0701] Entering user information

[0702] Users use devices such as smartphones or smart glasses to enter basic information such as age, gender, height, weight, skin tone, and budget, which is collected through applications installed on the devices.

[0703] Register your face photo and beauty style

[0704] Next, the user captures a photo of their face and inputs their desired beauty style information. For example, they can select specific makeup and hairstyles, such as natural makeup or a bob cut. This photo and beauty style information are also collected by the device.

[0705] Sending information

[0706] The collected basic information, face photo, and desired beauty style information are compiled in JSON format and sent to the server, which then receives the user's information and begins analyzing it.

[0707] AI-powered analysis and recommendation generation

[0708] The server analyzes the received user information using an artificial intelligence algorithm. The algorithm used here includes, for example, a generative AI model using deep learning. Based on this analysis, the server generates optimal fashion, makeup, and hairstyle suggestions for the user.

[0709] View Suggestions

[0710] The generated suggestions are sent from the server to the user's device, where they are displayed visually. The user can review the suggestions and, if they like them, click the button to display more information.

[0711] Providing supplier information

[0712] Information on where to purchase the suggested fashion and beauty items and beauty salons is also displayed on the device. Users can review this information and click on the purchase or reservation link to be redirected to the purchase website or salon's reservation system.

[0713] Real-time proposals in physical stores

[0714] In-store use allows users to use smart glasses or smartphones to receive real-time fashion and beauty recommendations, allowing them to try on suggested styles in the fitting room or cosmetics section.

[0715] Hardware and software used

[0716] The hardware used includes smartphones, smart glasses, cameras, etc. The software includes Python, OpenCV, and the requests library, which are combined to input, send, analyze, generate suggestions, and display information.

[0717] Specific examples

[0718] For example, suppose a 35-year-old female user enters her age, gender, height 160cm, weight 55kg, skin color light beige, and budget 20,000 yen. A photo of her face is captured, and her desired style of natural makeup and bob cut is selected and sent to the server. The server analyzes the data and suggests a style of autumn casual fashion, natural makeup, and bob cut. These suggestions are displayed in real time on the user's smart glasses, and the user can try them out in the store.

[0719] Prompt Sentence Examples

[0720] "Enter basic information such as your age, gender, height, weight, skin tone, and budget. Then, capture a photo of your face and select the makeup and hairstyle style you want. The AI ​​will then suggest the best fashion, makeup, and hairstyle for you."

[0721] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0722] Step 1: Enter your user information

[0723] Users use a smartphone or smart glasses to input basic information such as age, gender, height, weight, skin color, and budget. The input information is collected by an application installed on the device. This basic information becomes input data. The input data is packaged in JSON format and sent to the server.

[0724] Step 2: Register your face photo and beauty style

[0725] The user captures a photo of their face using the device's camera and inputs the desired makeup and hairstyle. For example, they can select a style such as natural makeup or a bob cut. This data is also compiled in JSON format and sent to the server. In this step, the face photo and beauty style information become input data and are sent to the server.

[0726] Step 3: Analyze the information and generate recommendations

[0727] The server uses an artificial intelligence algorithm to analyze the received data based on the basic information, face photo, and beauty style information. A generative AI model is used in this process. Specifically, based on the input data, the system extracts data features and performs statistical inference to suggest the most suitable fashion and beauty styles for users of the same age. The output is a recommendation of the most suitable fashion, makeup, and hairstyle for the user.

[0728] Step 4: Submit your proposal

[0729] The server generates and sends the proposed results to the user's device. The proposed results consist of both visual (images and graphics) and text formats. This output data is sent to the user's device and received by the application.

[0730] Step 5: Viewing the Suggestion Results

[0731] The user's device displays the received proposal results on the application's user interface. The user can check the displayed proposals and click a button to display detailed information. The input data for this step are the proposal results sent from the server, and the output data is the content displayed on the user interface.

[0732] Step 6: Provide supplier and beauty salon information

[0733] The user's device displays the suggested fashion and beauty item supplier information and beauty salon information to the user. The user can confirm this information and click on the purchase or reservation link to be redirected to the supplier website or salon reservation system. The input data is the supplier information and beauty salon information included in the recommendation results, and the output data is the redirected web page.

[0734] Step 7: Real-time in-store recommendations

[0735] When a user uses smart glasses or a smartphone in a physical store, the device displays appropriate fashion and beauty suggestions in real time. The user can try on the suggested styles in the store. Input data includes the user's current location in the store and a real-time facial photo, and the output data is the suggestions that are best suited to the current situation.

[0736] Step 8: Recording economic behavior data

[0737] The server records the user's purchasing and salon reservation behavior and stores it in an economic behavior database. This data is used to analyze the user's preferences and behavioral patterns. The input data are the purchasing and reservation actions, and the output data is the updated economic behavior database.

[0738] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0739] The present invention relates to a system that combines a user's basic information, facial photo, desired beauty style information, and an emotion engine that recognizes the user's emotions. This makes it possible to provide optimal fashion, makeup, and hairstyle suggestions while taking the user's emotions into consideration. The specific configuration and operation of the system are described below.

[0740] System configuration and operation

[0741] 1. Enter and submit user information

[0742] The user launches the application and enters basic information such as age, gender, height, weight, skin color, budget, etc. The device packages the entered basic information in JSON format and sends it to the server.

[0743] 2. Register your face photo and enter your beauty style

[0744] The user then uploads a photo of their face and selects the desired makeup and hairstyle, which is also compiled into JSON format by the device and sent to the server.

[0745] 3. Emotion Recognition by Emotion Engine

[0746] The server analyzes the received facial photos and uses an emotion engine to recognize the user's emotions, which are then used to adjust fashion and beauty suggestions.

[0747] 4. Comprehensive production by AI

[0748] The server analyzes the received basic information, emotional information, facial photo, and desired beauty style information, and uses an artificial intelligence algorithm to comprehensively produce the most suitable fashion, makeup, and hairstyle for the user. This information is generated as an integrated visual proposal, which is adjusted based on the emotional information. The proposal results are sent from the server to the device.

[0749] 5. Confirmation of proposal and provision of further information

[0750] The device analyzes the recommendations received from the server and displays visual suggestions on the app's interface. The user can review the suggestions and, if they like them, click the "View More Information" button. This will display information about where to purchase the suggested fashion and beauty items and beauty salons.

[0751] 6. Purchase and reservation assistance

[0752] If the user wishes to purchase the suggested item or make a salon reservation, they operate the device and click on the link, which redirects the user to the purchase website or salon reservation system. The server records this action and stores it in the economic behavior database.

[0753] 7. Providing advice on skin type and hair problems

[0754] Additionally, users can input their skin type and hair concerns. The device sends this information to the server, which uses AI to generate appropriate advice and a list of related products. Emotional information is taken into account to provide tailored advice. The device then displays the received advice and product list to the user.

[0755] Specific examples

[0756] Example 1: Entering basic information and beauty information

[0757] A 35-year-old female user logs into the app and enters her age, gender, height 160cm, weight 55kg, skin color light beige, and budget of 20,000 yen. Next, she uploads a photo of her face and selects her desired style of natural makeup and bob cut. The device compiles this information into JSON format and sends it to the server.

[0758] Example 2: Emotion Recognition with an Emotion Engine

[0759] The server analyzes the received facial photo, and the emotion engine recognizes the user's emotion as "happy." This emotion information is used to tailor the suggestions.

[0760] Example 3: Proposal generation and display

[0761] Based on the input basic information, desired style information, and emotional information, the server generates style suggestions, such as autumn casual fashion, natural makeup, and bob cuts, suitable for women of the same age. The results of these suggestions are sent to the device as visuals. The device displays the suggestions to the user, who can click a button to view more information.

[0762] Example 4: Purchasing and reservation assistance

[0763] The user likes the suggested items, a blouse and a skirt, and clicks on a link to purchase them. The device redirects the user to the website where the items are purchased, and the server records this action.

[0764] In this way, the system based on the present invention provides users with customized fashion and beauty suggestions that take emotional information into consideration, creating an environment where they can make purchases and reservations quickly.

[0765] The processing flow will be explained below.

[0766] Program processing steps

[0767] Step 1: Enter your user information

[0768] Behavior 1.1

[0769] Users launch the app and enter basic information such as age, gender, height, weight, skin color, and budget into a form within the app.

[0770] Behavior 1.2

[0771] The device packages the basic information entered in JSON format and sends it to the server.

[0772] Step 2: Register your face photo and enter your beauty style

[0773] Behavior 2.1

[0774] Users can upload a photo of their face to the app using their device's camera or by selecting a suitable photo from their gallery.

[0775] Behavior 2.2

[0776] The user selects the desired makeup and hairstyle (e.g., natural makeup, bob cut).

[0777] Behavior 2.3

[0778] The device compiles the facial photo and beauty style information into JSON format and sends it to the server.

[0779] Step 3: Emotion recognition by the emotion engine

[0780] Behavior 3.1

[0781] The server analyzes the received facial photo using an emotion engine to recognize the user's emotions.

[0782] Behavior 3.2

[0783] The server stores the recognized emotion information in a database and prepares it for further processing.

[0784] Step 4: Comprehensive production by AI

[0785] Behavior 4.1

[0786] The server launches an artificial intelligence algorithm to analyze basic information, emotional information, facial photos, and desired beauty style information.

[0787] Behavior 4.2

[0788] The server's artificial intelligence algorithm generates optimal fashion, makeup, and hairstyle suggestions based on the original input information and emotional information.

[0789] Behavior 4.3

[0790] The server packages the generated proposal results in JSON format and sends them to the terminal.

[0791] Step 5: Review your proposal and provide further information

[0792] Operation 5.1

[0793] The device analyzes the suggestions received from the server and displays visual suggestions on the app interface.

[0794] Behavior 5.2

[0795] The user reviews the suggestions and, if they like them, clicks the button to display more information.

[0796] Operation 5.3

[0797] The device displays a list of suggested fashion and beauty items, as well as information on where to buy them and beauty salons.

[0798] Step 6: Purchase and reservation assistance

[0799] Operation 6.1

[0800] If the user wishes to purchase the suggested item or make a salon appointment, they click on the displayed link.

[0801] Operation 6.2

[0802] Based on the link clicked, the device redirects the user to the purchase website or beauty salon reservation system.

[0803] Operation 6.3

[0804] The server records the user's purchasing and reservation behavior and stores it in an economic behavior database.

[0805] Step 7: Providing advice on skin type and hair concerns

[0806] Operation 7.1

[0807] Users can also enter their skin type and hair concerns (e.g., dry skin, dandruff, etc.) into the app.

[0808] Operation 7.2

[0809] The device packages the entered information about skin type and hair concerns in JSON format and sends it to the server.

[0810] Operation 7.3

[0811] The server uses artificial intelligence algorithms to generate appropriate advice and a list of related products based on the input skin type and hair concerns, while also taking emotional information into account.

[0812] Operation 7.4

[0813] The server packages the generated advice and product list in JSON format and sends it to the terminal.

[0814] Operation 7.5

[0815] The device displays the received advice and a list of related products on the app interface.

[0816] Specific examples

[0817] Example 1: Entering basic information and beauty information

[0818] A 35-year-old female user logs into the app and enters her age, gender, height 160cm, weight 55kg, skin color light beige, and budget of 20,000 yen. Next, she uploads a photo of her face and selects her desired style of natural makeup and bob cut. The device compiles this information into JSON format and sends it to the server.

[0819] Example 2: Emotion Recognition with an Emotion Engine

[0820] The server analyzes the received facial photo, and the emotion engine recognizes the user's emotion as "happy." This emotion information is used to tailor the suggestions.

[0821] Example 3: Proposal generation and display

[0822] Based on the input basic information, desired style information, and emotional information, the server generates style suggestions, such as autumn casual fashion, natural makeup, and bob cuts, suitable for women of the same age. The results of these suggestions are sent to the device as visuals. The device displays the suggestions to the user, who can click a button to view more information.

[0823] Example 4: Purchasing and reservation assistance

[0824] The user likes the suggested items, a blouse and a skirt, and clicks on a link to purchase them. The device redirects the user to the website where the items are purchased, and the server records this action.

[0825] By following these steps, the system based on the present invention provides users with customized fashion and beauty suggestions that take emotional information into account, creating an environment for quick purchases and reservations.

[0826] Example 2

[0827] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0828] Current beauty and fashion recommendation systems often do not adequately consider users' emotions or detailed personal information. As a result, the recommendations provided may not meet users' expectations. Furthermore, many systems do not include a means to smoothly complete purchase or reservation procedures, resulting in a lack of convenience for users. This poses a challenge, resulting in a decrease in user satisfaction.

[0829] The identification process by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes: means for inputting basic information about the user; means for inputting a facial photo and desired beauty style information about the user; artificial intelligence algorithm means for analyzing the input information and making optimal fashion, makeup, and hairstyle suggestions; emotion recognition means for analyzing the received facial photo and recognizing the user's emotions; means for generating suggestions and adjusting them based on the emotion information; means for providing information about where to purchase the suggested fashion and beauty items and beauty salons; and means for providing appropriate advice and a list of related products based on the user's skin type or hair concerns. This makes it possible to provide more personalized suggestions that take into account the user's emotions and detailed personal information, facilitating smoother purchasing and reservation procedures.

[0830] "Basic information" refers to personal information such as the user's age, gender, height, weight, skin color, and budget.

[0831] "Face Photo" refers to an image of a user's face uploaded by the user.

[0832] "Beauty style information" refers to information about the user's desired beauty style, such as desired makeup and hairstyle.

[0833] "Artificial intelligence algorithm" refers to an algorithm that analyzes input information and generates optimal fashion suggestions, makeup suggestions, and hairstyle suggestions.

[0834] "Emotion recognition means" refers to a system that analyzes the received facial photo and recognizes the user's emotions.

[0835] The "means for adjusting the recommendation result" refers to a means for adjusting the generated recommendation result based on the recognized emotion.

[0836] "Purchaser Information" refers to information about where to purchase suggested fashion and beauty items.

[0837] "Beauty salon information" refers to information about a beauty salon that provides the proposed beauty services.

[0838] "Skin type" refers to the nature and condition of the user's skin.

[0839] "Hair concerns" refers to the hair-related problems and hopes that users have.

[0840] "Advice means" refers to a means for providing appropriate advice based on the user's skin type and hair concerns.

[0841] "Related Product Listings" refers to lists of beauty products or items recommended based on the advice provided.

[0842] "Means for inputting basic information" refers to the interface and functions that allow users to input basic information.

[0843] "Means for inputting face photo and beauty style information" refers to an interface or function that allows a user to input a face photo and desired beauty style information.

[0844] The present invention relates to a system that integrates a user's basic information, facial photo, desired beauty style information, and emotional information to provide optimal fashion, makeup, and hairstyle suggestions. This system can personalize suggestions using an emotion engine that recognizes the user's emotions and an artificial intelligence algorithm. The specific configuration and operation of the present invention are described below.

[0845] composition

[0846] 1. Enter and submit user information

[0847] The user launches the application and enters basic information such as age, gender, height, weight, skin color, and budget. The device packages the entered basic information in JSON format and sends it to the server. The software used is an interface for entering basic information and a program for converting it to JSON format.

[0848] 2. Register your face photo and enter your beauty style

[0849] The user then uploads a photo of their face and selects the desired makeup and hairstyle. This is also compiled into JSON format by the device and sent to the server. The software used is an interface for uploading face photos and selecting beauty styles, as well as a comprehensive data package program.

[0850] 3. Emotion Recognition by Emotion Engine

[0851] The server analyzes the received facial photos and recognizes the user's emotions using an emotion engine. This emotion information is used to adjust fashion and beauty suggestions. The hardware used is a high-performance image analysis server, and the software is an emotion recognition engine (e.g., Google Cloud Vision API).

[0852] 4. Comprehensive production by AI

[0853] The server analyzes the received basic information, emotional information, facial photo, and desired beauty style information, and uses artificial intelligence algorithms to comprehensively produce the most suitable fashion, makeup, and hairstyle for the user. This information is generated as an integrated visual proposal, which is adjusted based on the emotional information. The proposal results are sent from the server to the device. The software used is a machine learning model (e.g., TensorFlow) and an image generation algorithm.

[0854] 5. Confirmation of proposal and provision of further information

[0855] The device analyzes the recommendation results received from the server and displays visual suggestions on the app's interface. The user checks the suggestions and clicks the display button if more information is needed. This displays information on where to purchase the suggested fashion and beauty items and beauty salons. The software used is the recommendation result display interface and the detailed information display program.

[0856] 6. Purchase and reservation assistance

[0857] If the user wishes to purchase the suggested item or make a salon reservation, they operate the device and click on the link. This causes the device to redirect the user to the purchasing website or salon reservation system. The server records this behavior and stores it in an economic behavior database. The software used is a redirect program and a behavior recording program.

[0858] 7. Providing advice on skin type and hair problems

[0859] Additionally, users can enter their skin type and hair concerns. The device sends this information to a server, which uses AI to generate appropriate advice and a list of related products. Emotional information is taken into account to provide tailored advice. The device then displays the received advice and product list to the user. The software used is a skin type and hair concern input interface and a program that generates a list of related products.

[0860] Specific examples

[0861] Example 1: Entering basic information and beauty information

[0862] A 35-year-old female user logs into the app and enters her age, gender, height 160cm, weight 55kg, skin color light beige, and budget of 20,000 yen. Next, she uploads a photo of her face and selects her desired style of natural makeup and bob cut. The device compiles this information into JSON format and sends it to the server.

[0863] Example 2: Emotion Recognition with an Emotion Engine

[0864] The server analyzes the received facial photo, and the emotion engine recognizes the user's emotion as "happy." This emotion information is used to tailor the suggestions.

[0865] Example 3: Proposal generation and display

[0866] Based on the user's basic information, desired style, and emotional information, the server proposes styles suitable for women of the same age, such as autumn casual fashion, natural makeup, and bob cuts. The proposed results are sent to the device as visuals. The device displays the proposed content to the user, who can click a button to view more information.

[0867] Example 4: Purchasing and reservation assistance

[0868] The user likes the suggested items, a blouse and a skirt, and clicks on a link to purchase them. The device redirects the user to the website where the items are purchased, and the server records this action.

[0869] Prompt Sentence Examples

[0870] "Please suggest something that suits the casual fashion and natural makeup worn by a 35-year-old woman. Furthermore, the user is showing the emotion 'happy.'"

[0871] This allows users to get the best fashion and beauty suggestions from the generative AI model.

[0872] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0873] Step 1:

[0874] A user launches the application and logs in. They then enter basic information such as age, gender, height, weight, skin tone, and budget.

[0875] Input: Basic information entered by the user.

[0876] Specific operation: User authentication is performed on the application login screen, and after moving to the main screen, a basic information input form is displayed. When the user clicks the submit button, the input data is converted to JSON format on the terminal side.

[0877] Output: Basic information converted to JSON format.

[0878] Step 2:

[0879] The terminal transmits the converted basic information to the server.

[0880] Input: Basic information in JSON format.

[0881] Specific operation: The device sends JSON data to the server using the REST API.

[0882] Output: Basic information data received by the server.

[0883] Step 3:

[0884] Users upload a photo of their face and select the desired makeup and hairstyle.

[0885] Input: A photo of the user's face and desired beauty style.

[0886] Specific operation: A face photo upload screen is displayed, the user selects a photo, clicks the upload button, and after selecting a beauty style, the device compiles this information into JSON format.

[0887] Output: A JSON-formatted face photo and desired beauty style information.

[0888] Step 4:

[0889] The device sends a facial photo and beauty style information in JSON format to the server.

[0890] Input: Face photo and beauty style information in JSON format.

[0891] Specific operation: The device sends data to the server using the REST API.

[0892] Output: Face photo and beauty style information data received by the server.

[0893] Step 5:

[0894] The server analyzes the received facial photograph and recognizes the user's emotions using an emotion recognition engine.

[0895] Input: Facial photo data.

[0896] Specific operation: The server uses a high-performance image analysis server to run a facial recognition algorithm and analyzes emotions using an emotion recognition engine (e.g., Google Cloud Vision API).

[0897] Output: Emotion information data.

[0898] Step 6:

[0899] The server analyzes basic information, emotional information, facial photos, and desired beauty style information, and uses artificial intelligence algorithms to produce the optimal fashion, makeup, and hairstyle.

[0900] Input: Basic information, emotional information, face photo, desired beauty style information.

[0901] How it works: The server analyzes this information using a data analysis module and generates suggestions using a machine learning model (e.g., TensorFlow), which are then visualized using an image generation algorithm.

[0902] Output: Visual suggestion data.

[0903] Step 7:

[0904] The server transmits the visual suggestion data to the terminal.

[0905] Input: Visual suggestion data.

[0906] Specific operation: The server sends visual suggestion data to the device using the REST API.

[0907] Output: Visual suggestion data received by the device.

[0908] Step 8:

[0909] The device analyzes the received suggestions and displays visual suggestions on the app interface. The user can review the suggestions and click the view button to view more information.

[0910] Input: Visual suggestion data.

[0911] What happens: The device parses the received content and displays it in the app's UI as images and text. When the user clicks the details button, the additional information expands to show more information.

[0912] Output: Visual suggestions and detailed information displayed in the app's UI.

[0913] Step 9:

[0914] If the user wishes to purchase the suggested item or make a salon reservation, he or she operates the terminal and clicks on the link.

[0915] Input: Purchase or reservation link.

[0916] What happens: When a user clicks on a purchase link, their device's browser opens to the website where the purchase is made.

[0917] Output: Purchasing website or salon booking system.

[0918] Step 10:

[0919] The server records user behavior and stores it in an economic behavior database.

[0920] Input: User behavior data.

[0921] Specific operation: The server logs users' link clicks and purchase behavior and stores them in an economic behavior database.

[0922] Output: Stored behavioral data.

[0923] Step 11:

[0924] The user enters additional information such as their skin type and hair concerns, and the device sends this to the server.

[0925] Input: Skin type and hair concerns.

[0926] Specific behavior: When the user fills out the additional information input form and clicks the submit button, the device converts the data into JSON format and sends it to the server.

[0927] Output: Skin type and hair problem information received by the server.

[0928] Step 12:

[0929] The server uses AI to generate appropriate advice and a list of related products based on skin type and hair concerns.

[0930] Input: Skin type and hair concerns.

[0931] How it works: The server uses an AI model to analyze the input information and generate advice and product lists.

[0932] Output: Advice and related product list.

[0933] Step 13:

[0934] The terminal displays the received advice and related product list to the user.

[0935] Input: Advice and related product listings.

[0936] Specific behavior: The device parses the received content and displays it in the app's UI.

[0937] Output: Advice and related product list displayed in the app UI.

[0938] (Application example 2)

[0939] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0940] Today's consumers demand personalized fashion and beauty recommendations, but existing systems often only offer one-size-fits-all recommendations, resulting in low satisfaction. Furthermore, the effectiveness of recommendations is limited because they are not tailored based on real-time emotional information. Furthermore, there is a lack of support for in-store purchases and try-ons, creating a need for ways to improve the user shopping experience.

[0941] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes: means for inputting basic information about the user; means for inputting a facial photo and desired beauty style information about the user; means including an artificial intelligence algorithm for analyzing the input information and making optimal fashion, makeup, and hairstyle suggestions; means for providing information on where to purchase the suggested fashion and beauty items and beauty salon information; means for providing appropriate advice and a list of related products based on the user's skin type or hair concerns; means for displaying the suggestion results on smart glasses worn by the user in a physical store, recognizing emotions in real time, and adjusting the suggestion content based on the emotional information; and means for supporting the user in trying on and purchasing items they like through the smart glasses. This enables the provision of personalized suggestions and adjustments based on real-time emotional information, as well as an improved shopping experience in a physical store.

[0942] "Means for inputting basic user information" refers to devices or applications for inputting basic information such as the user's age, gender, height, weight, skin color, and budget.

[0943] The "means for inputting the user's facial photograph and desired beauty style information" refers to a device or application for inputting the user's facial photograph and beauty style information such as desired makeup and hairstyle.

[0944] "Means including an artificial intelligence algorithm for analyzing input information and making optimal suggestions for fashion, makeup, and hairstyle" refers to systems or software that include an artificial intelligence algorithm used to analyze input user information and make suggestions for optimal fashion, makeup, and hairstyle.

[0945] The "means for providing information on where to purchase suggested fashion and beauty items and information on beauty salons" refers to a system or application for providing information on where to purchase suggested fashion and beauty items and information on beauty salons.

[0946] "Means for providing appropriate advice and a list of related products based on the user's skin type or hair concerns" refers to a system or software for providing appropriate advice and a list of related products based on the user's skin type or hair concerns.

[0947] "A means for displaying the proposal results on smart glasses worn by the user in a physical store, recognizing emotions in real time, and adjusting the proposal content based on that emotional information" refers to a system or algorithm for displaying the proposal results on smart glasses worn by the user in a physical store, recognizing the user's emotions in real time, and adjusting the proposal content based on that emotional information.

[0948] "Means for supporting users in trying on and purchasing items they like through smart glasses" refers to a system or software that supports users in trying on and purchasing items they like through the UI of smart glasses.

[0949] The present invention relates to a system that combines a user's basic information, facial photo, desired beauty style information, and an emotion engine that recognizes the user's emotions. This makes it possible to provide optimal fashion, makeup, and hairstyle suggestions while taking the user's emotions into consideration. The specific configuration and operation of the system are described below.

[0950] System configuration and operation

[0951] 1. Enter and submit user information

[0952] Users wear smart glasses and input basic information such as age, gender, height, weight, skin color, and budget. The smart glasses package the input basic information in JSON format and send it to the server. The smart glasses are equipped with a camera and touchpad, making it easy for users to input information.

[0953] 2. Register your face photo and enter your beauty style

[0954] The user then takes a photo of their face using the smart glasses' camera and selects the desired makeup and hairstyle, which is then compiled into JSON format by the smart glasses and sent to the server.

[0955] 3. Emotion Recognition by Emotion Engine

[0956] The server analyzes the received facial photo and uses an emotion engine (emotion recognition technology such as DeepFace) to recognize the user's emotions. This emotional information is used to adjust fashion and beauty suggestions.

[0957] 4. Comprehensive production by AI

[0958] The server analyzes the received basic information, emotional information, facial photo, and desired beauty style information, and then uses an artificial intelligence algorithm (e.g., a model using Keras) to comprehensively produce the most suitable fashion, makeup, and hairstyle for the user. This information is then generated as an integrated visual suggestion, which is adjusted based on the emotional information. The suggestion results are sent from the server to the smart glasses.

[0959] 5. Confirmation of proposal and provision of further information

[0960] The smart glasses analyze the recommendations received from the server and display visual suggestions to the user. The user reviews the recommendations and, if they like them, clicks the display details button on the touchpad. This displays information on where to purchase the suggested fashion and beauty items and beauty salons on the smart glasses' display.

[0961] 6. Purchase and reservation assistance

[0962] If the user chooses a suggested item and wishes to purchase or try it on, they can operate the smart glasses' interface and click the corresponding button, which will redirect the user to the purchase website or salon's reservation system. The server records this action and stores it in the economic behavior database.

[0963] 7. Providing advice on skin type and hair problems

[0964] Additionally, users can input their skin type and hair concerns through the smart glasses. The smart glasses then send this information to the server, which uses AI to generate appropriate advice and a list of related products. Emotional information is taken into account to provide tailored advice. The smart glasses then display the received advice and product list to the user.

[0965] Specific examples

[0966] Example 1: Entering basic information and beauty information

[0967] A 30-year-old female user puts on smart glasses and enters her age, gender, height 160cm, weight 55kg, skin color light beige, and budget of 20,000 yen. Next, she takes a photo of her face with the smart glasses' camera and selects her desired style of natural makeup and short hair. The smart glasses then compile this information into JSON format and send it to the server.

[0968] Example 2: Emotion Recognition with an Emotion Engine

[0969] The server analyzes the received facial photo and uses an emotion engine (DeepFace) to recognize the user's emotion as "happy." This emotion information is used to adjust the suggestions.

[0970] Example 3: Proposal generation and display

[0971] Based on the input basic information, desired style information, and emotional information, the server generates style suggestions, such as autumn casual fashion, natural makeup, and short haircuts, suitable for women of the same age. The results of these suggestions are sent as visuals to the smart glasses. The smart glasses then display the suggestions to the user, who can then operate the touchpad to view more information.

[0972] Example 4: Purchasing and reservation assistance

[0973] The user likes the suggested items, a jacket and pants, and clicks on a link to purchase them. The smart glasses redirect the user to the website where they purchased the items, and the server records this action.

[0974] Prompt Sentence Examples

[0975] Recognize user emotions and provide customized fashion and beauty recommendations based on the following information:

[0976] Age: 30

[0977] Gender: Female

[0978] Height: 160cm

[0979] Weight: 55kg

[0980] Skin Tone: Light Beige

[0981] Budget: 20,000 yen

[0982] Face photo: face.jpg

[0983] Suggest the best style based on user sentiment.

[0984] In this way, the system based on the present invention provides users with customized fashion and beauty suggestions that take emotional information into account, allowing for real-time adjustments and quick purchase and try-on in-store.

[0985] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0986] Step 1:

[0987] The user puts on the smart glasses and inputs basic information, such as age, gender, height, weight, skin tone, and budget, using the touchpad on the smart glasses interface. The input basic information is converted into JSON format and sent to the server, where the user information is stored and available for subsequent analysis.

[0988] Step 2:

[0989] The user takes a photo of their face using the smart glasses' camera and inputs their desired beauty style information (e.g., type of makeup, hairstyle). The smart glasses also compile this information into JSON format and send it to the server. The server then receives the user's face photo and desired style information, and is ready for emotion recognition.

[0990] Step 3:

[0991] The server analyzes the received facial photo and recognizes the user's emotions using an emotion engine (such as DeepFace). Specifically, it uses the DeepFace library to extract facial features from the facial photo and determine the emotion based on those features. The emotional information obtained here is then used to adjust the fashion suggestions.

[0992] Step 4:

[0993] The server combines the user's basic information, face photo, desired beauty style information, and emotional information received from the user and analyzes it using an artificial intelligence algorithm (e.g., a generative AI model using Keras). This analysis generates optimal fashion, makeup, and hairstyle recommendations. The AI ​​converts the user's information into personalized recommendations based on statistical data and trend information.

[0994] Step 5:

[0995] The server sends the generated suggestions to the smart glasses worn by the user in the physical store. The smart glasses display the suggestions to the user. The user can review the suggestions using the touchpad, and if they like them, they can access more detailed information by tapping the display button for more information.

[0996] Step 6:

[0997] If the user likes any of the suggested fashion items, they will be assisted in trying them on and purchasing them through the smart glasses' UI. Specifically, the smart glasses will provide fitting room reservations and detailed product information based on the user's preferences, and guide the user through the operations required for trying on and purchasing. This allows the user to quickly try on and purchase items using the smart glasses.

[0998] Step 7:

[0999] The server records the user's economic behavior data and stores it in a database. Specifically, it collects and records behavioral data such as which offers the user accepted, which items they purchased, and which beauty salons they booked. This data can be used for subsequent analysis and to improve the accuracy of future offers.

[1000] Step 8:

[1001] Users also input their skin type and hair concerns through the smart glasses. The smart glasses send this information to a server, which uses AI to generate appropriate advice and a list of related products. The advice takes emotional information into account, and is received by the smart glasses and displayed to the user.

[1002] Through these steps, users can receive personalized fashion and beauty recommendations based on their emotions, with real-time adjustments and a quick in-store purchasing experience.

[1003] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1004] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1005] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[1006] [Third embodiment]

[1007] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[1008] 5, the data processing system 310 includes the data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[1009] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1010] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[1011] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1012] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1013] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1014] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1015] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1016] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1017] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1018] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[1019] The present invention relates to a system that utilizes artificial intelligence to provide a means for users to input their basic information, a photograph of their face, and information about their desired beauty style, and then analyzes this information to provide comprehensive fashion, makeup, and hairstyle suggestions. This system automatically provides optimal fashion and beauty suggestions by having users input the necessary information.

[1020] System configuration and operation

[1021] 1. Enter and submit user information

[1022] The user launches the application and enters basic information such as age, gender, height, weight, skin tone, budget, etc. The device packages the entered information in JSON format and sends it to the server.

[1023] 2. Register your face photo and beauty style

[1024] The user then uploads a photo of their face and selects the desired makeup and hairstyle, which is also compiled into JSON format by the device and sent to the server.

[1025] 3. AI-generated recommendations

[1026] Based on the received information, the server uses an artificial intelligence algorithm to comprehensively produce the most suitable fashion, makeup, and hairstyle for the user. This information is analyzed and an integrated visual proposal is generated. The proposal results are then sent from the server to the device.

[1027] 4. Review the proposal and view more information

[1028] The device receives the recommendations from the server and displays them to the user on the app interface. The user can review the recommendations and, if they like them, click the "Display details" button. This will display information about where to purchase the recommended fashion and beauty items and beauty salons.

[1029] 5. Purchase and reservation assistance

[1030] If the user wishes to purchase the suggested item or make a salon reservation, they operate the device and click on the link, which redirects the user to the purchase website or salon reservation system. The server records this action and stores it in the economic behavior database.

[1031] 6. Providing advice on skin type and hair problems

[1032] Additionally, users can input their skin type and hair concerns. The device sends this information to the server, which uses AI to generate appropriate advice and a list of related products and send them to the device. The device then displays the received advice and product list to the user.

[1033] Specific examples

[1034] Example 1: Entering basic information and beauty information

[1035] A 35-year-old female user logs into the app and enters her age, gender, height 160cm, weight 55kg, skin color light beige, and budget of 20,000 yen. Next, she uploads a photo of her face and selects her desired style of natural makeup and bob cut. The device compiles this information into JSON format and sends it to the server.

[1036] Example 2: Proposal generation and display

[1037] The server analyzes the input information and generates style suggestions, such as autumn casual fashion, natural makeup, and bob cuts, suitable for women of the same age. These suggestions are sent as visuals to the device. The device displays the suggestions to the user, who can click a button to view more information.

[1038] Example 3: Purchasing and reservation assistance

[1039] The user likes the suggested items, a blouse and a skirt, and clicks on a link to purchase them. The device redirects the user to the website where the items are purchased, and the server records this action.

[1040] In this way, the system based on the present invention provides users with quick and accurate comprehensive fashion and beauty suggestions, and smoothly links these to purchases and service use.

[1041] The processing flow will be explained below.

[1042] Program processing steps

[1043] Step 1: Enter your user information

[1044] Behavior 1.1

[1045] Users launch the app and enter basic information such as age, gender, height, weight, skin color, and budget into a form within the app.

[1046] Behavior 1.2

[1047] The device packages the basic information entered in JSON format and sends it to the server.

[1048] Step 2: Register your face photo and enter your beauty style

[1049] Behavior 2.1

[1050] Users can upload a photo of their face to the app using their device's camera or by selecting a suitable photo from their gallery.

[1051] Behavior 2.2

[1052] The user selects the desired makeup and hairstyle (e.g., natural makeup, bob cut).

[1053] Behavior 2.3

[1054] The device compiles the facial photo and beauty style information into JSON format and sends it to the server.

[1055] Step 3: Comprehensive production by AI

[1056] Behavior 3.1

[1057] The server then calls an artificial intelligence algorithm to analyze the received basic information, facial photo, and desired beauty style information.

[1058] Behavior 3.2

[1059] The server's artificial intelligence algorithm generates optimal fashion, makeup, and hairstyle suggestions based on the user's age, gender, height, weight, skin color, budget, and desired style information.

[1060] Behavior 3.3

[1061] The server packages the generated proposal results in JSON format and sends them to the terminal.

[1062] Step 4: Review your proposal and provide further information

[1063] Behavior 4.1

[1064] The device analyzes the suggestions received from the server and displays visual suggestions on the app interface.

[1065] Behavior 4.2

[1066] The user reviews the suggestions and, if they like them, clicks the button to display more information.

[1067] Behavior 4.3

[1068] The device displays a list of suggested fashion and beauty items, as well as information on where to purchase them and beauty salons.

[1069] Step 5: Purchase and reservation assistance

[1070] Operation 5.1

[1071] If the user wishes to purchase the suggested item or make a salon appointment, they click on the displayed link.

[1072] Behavior 5.2

[1073] Based on the link clicked, the device redirects the user to the purchase website or beauty salon reservation system.

[1074] Operation 5.3

[1075] The server records the user's purchasing and reservation behavior and stores it in an economic behavior database.

[1076] Step 6: Providing advice on skin type and hair concerns

[1077] Operation 6.1

[1078] Users can also enter their skin type and hair concerns (e.g., dry skin, dandruff, etc.) into the app.

[1079] Operation 6.2

[1080] The device packages the entered information about skin type and hair concerns in JSON format and sends it to the server.

[1081] Operation 6.3

[1082] The server uses artificial intelligence algorithms to generate appropriate advice and a list of related products based on the input skin type and hair concerns.

[1083] Operation 6.4

[1084] The server packages the generated advice and product list in JSON format and sends it to the terminal.

[1085] Operation 6.5

[1086] The device displays the received advice and a list of related products on the app interface.

[1087] ---

[1088] Through these steps, users can receive customized fashion and beauty suggestions through a consistent process, and can quickly make purchases or reservations.

[1089] Example 1

[1090] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1091] Conventional beauty and fashion recommendation systems have the problem that it takes a lot of time and effort for users to match their own information and preferences in detail. In addition, there are few systems that integrate various information to make comprehensive recommendations, and the purchasing and reservation process after the recommendations is often fragmented. This has led to issues such as low user satisfaction and a lack of improvement in system usage efficiency.

[1092] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1093] In this invention, the server includes means for inputting the user's personal information, means for inputting the user's face photo and desired beauty style information, means for packaging the input information into a data format and sending it to the server, means for analyzing the input information using an artificial intelligence algorithm and making optimal fashion, makeup, and hairstyle suggestions, means for displaying the suggestions received from the server to the user, means for providing information on where to purchase the suggested fashion and beauty items and information on beauty salons, and means for providing appropriate advice and a list of related products based on the user's skin type or hair concerns. This allows users to input their own information within a single system, receive optimal fashion and beauty suggestions based on that information, and also enable them to purchase related items and make beauty salon reservations in an integrated manner.

[1094] "User" means an individual who uses the System.

[1095] "Personal information" refers to information such as age, gender, height, weight, skin color, and budget that users enter into the system.

[1096] "Facial photo" refers to image data of a user's face that the user uploads to the system.

[1097] "Beauty style information" refers to information about the makeup and hairstyle style desired by the user.

[1098] A "data format" refers to a format that structures information to make it easier to store and communicate; a commonly used example is the JSON format.

[1099] "Server" refers to the central computer of the system, a computing resource that receives user input, analyzes it using artificial intelligence algorithms, and makes suggestions.

[1100] "Artificial intelligence algorithm" refers to machine learning and deep learning models and methods used to analyze input data and generate optimal fashion, makeup, and hairstyle suggestions.

[1101] "Suggestion results" is a general term for fashion suggestions, makeup suggestions, and hairstyle suggestions that are analyzed by an artificial intelligence algorithm and provided to users.

[1102] "Purchaser Information" refers to information about stores and online stores where suggested fashion and beauty items can be purchased.

[1103] "Beauty salon information" refers to information such as the location and contact details of the beauty salon that offers the proposed beauty style.

[1104] "Skin type" refers to the condition or characteristics of the user's skin (e.g., dry skin, oily skin).

[1105] "Hair concerns" refers to problems with the user's hair or areas they would like to improve (e.g., thinning hair, split ends).

[1106] The "List of Related Products" refers to a list of beauty products recommended based on the user's skin type and hair concerns.

[1107] "Purchase Intent" refers to a state in which a user has indicated an intention to purchase or use a proposed item or service.

[1108] "Redirecting" refers to the act of sending a user to another web page or service.

[1109] "Economic behavior data" refers to behavioral data when users engage in economic activities such as purchases and reservations.

[1110] "Recording and storing" refers to keeping specific information in a database or storage so that it can be accessed later.

[1111] The present invention relates to a system that utilizes artificial intelligence to input a user's basic information, a facial photo, and desired beauty style information, and then analyzes the information to provide comprehensive fashion, makeup, and hairstyle suggestions. This system automatically provides optimal fashion and beauty suggestions by having the user input the necessary information. Specific embodiments are described in detail below.

[1112] Enter and submit user information

[1113] Users launch a dedicated application and enter their personal information, including age, gender, height, weight, skin color, and budget. This information is packaged in JSON format by the device and sent to the server. Users also upload a photo of their face and select their desired beauty style (e.g., natural makeup, bob cut). This information is also packaged in JSON format by the device and sent to the server.

[1114] AI-powered recommendation generation

[1115] The server analyzes the received user information using an artificial intelligence algorithm. This algorithm uses machine learning models such as TensorFlow and PyTorch. The server generates optimal fashion, makeup, and hairstyle suggestions for the user, renders them in a visual format, and sends them to the device. The analyzed and generated suggestions are then sent to the device as JSON-formatted data.

[1116] Displaying the proposed results and providing detailed information

[1117] The device visually displays the received suggestions on the dedicated application interface. Users can check the suggestions displayed and click the confirmation button for more information, which will display information on where to purchase the suggested fashion items and detailed information on beauty salons.

[1118] Purchase and reservation assistance

[1119] If the user wishes to purchase the suggested item or make a reservation at a beauty salon, they click on the provided link on their device. By clicking on the link, the device redirects the user to the product purchase site or beauty salon reservation system. The user's clicking behavior is recorded by the server and later stored in the economic behavior database.

[1120] Advice on skin and hair problems

[1121] Additionally, users can enter additional information about their skin type and hair concerns. The device sends this information to the server, which uses AI algorithms to generate a list of appropriate advice and related products. This list and advice is then sent to the device and displayed to the user.

[1122] Examples and prompts

[1123] Example 1: Entering basic information and beauty information

[1124] A 35-year-old female user logs into the app and enters her age, height 160cm, weight 55kg, skin color light beige, and budget of 20,000 yen. Next, she uploads a photo of her face and selects her desired style of natural makeup and bob cut. The device compiles this information into JSON format and sends it to the server.

[1125] Example 2: Proposal generation and display

[1126] The server analyzes the input information and generates recommendations for autumn casual fashion, natural makeup, and a bob cut that are ideal for a 35-year-old woman, and sends them to the device. The device displays the recommendations and provides more information when the user clicks the "More Info" button.

[1127] Prompt Sentence Examples

[1128] "35-year-old female, height 160cm, weight 55kg, skin tone light beige, budget 20,000 yen. Prefer natural makeup and a bob cut. Please provide suggestions for casual fall fashion and specific shopping information."

[1129] The above is a detailed description of the embodiment of the present invention. This system allows users to efficiently and easily receive suggestions for optimal fashion and beauty styles, and then smoothly take the next action, such as purchasing a product or making a reservation at a beauty salon, based on that information.

[1130] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1131] Step 1: Enter and submit user information

[1132] The user starts the application and enters personal information (age, sex, height, weight, skin color, budget, etc.) by typing this information into a dedicated form.

[1133] Input: Personal information entered by the user

[1134] The terminal packages the entered information into JSON format.

[1135] Data processing: Validate the information entered by the user and convert it to JSON format.

[1136] Output: User details in JSON format

[1137] The terminal sends the generated JSON data to the server via an HTTP POST request.

[1138] Step 2: Register your face photo and beauty style

[1139] Users upload a photo of their face and select the desired beauty style (e.g., natural makeup, bob cut). Specifically, they take a photo of their face using the camera function on their smartphone and upload it to the app.

[1140] Input: Uploaded face photo and selected beauty style information

[1141] The device then reassembles this information into JSON format.

[1142] Data processing: Convert the face photo into the appropriate image format (e.g., JPEG) and include beauty style information in the JSON data.

[1143] Output: JSON formatted face photo and beauty style information

[1144] The terminal sends the generated JSON data to the server via an HTTP POST request.

[1145] Step 3: Generate recommendations with AI

[1146] The server analyzes the received JSON-formatted user information, face photo, and beauty style information using an artificial intelligence model (e.g., TensorFlow, PyTorch).

[1147] Input: JSON format user information, face photo, beauty style information

[1148] The server inputs this information into a generative AI model to generate optimal fashion, makeup, and hairstyle suggestions.

[1149] Data calculation: Generative AI models analyze input information and generate optimal recommendations

[1150] Output: JSON format suggestion results (fashion suggestions, makeup suggestions, hairstyle suggestions)

[1151] The server renders the generated proposal results in a visual format and sends them to the terminal as JSON data.

[1152] Step 4: View the results of the proposal and provide more information

[1153] The device displays the received proposal results on the interface of a dedicated app. The proposals are visually easy to understand.

[1154] Input: JSON formatted proposal results

[1155] Output: Visual suggestion results shown to the user

[1156] The user checks the suggested content and clicks the "Display detailed information" button, which displays information about where to purchase the suggested fashion items and detailed information about beauty salons.

[1157] Step 5: Assist with purchases and reservations

[1158] If users want to purchase or book a featured item or beauty service, they click on a link within the app.

[1159] Input: User clicks

[1160] The terminal responds to the click and redirects the user to a purchasing website or a beauty salon reservation system.

[1161] Data calculation: Link click event and redirect handling

[1162] Output: The redirected web page

[1163] The server records the user's clicking behavior and stores it in an economic behavior database.

[1164] Step 6: Providing advice on skin type and hair concerns

[1165] Users can also enter their skin type and hair concerns (e.g., dry skin, thinning hair).

[1166] Input: Information about your skin type and hair concerns

[1167] The device compiles this information into JSON format and sends it to the server.

[1168] The server passes the received information to an AI algorithm to generate optimal advice and related product lists.

[1169] Data arithmetic: AI algorithms analyze information and generate advice

[1170] Output: JSON formatted advice and related product list

[1171] The terminal displays the received information again to the user, allowing the user to check appropriate advice and related products.

[1172] (Application example 1)

[1173] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1174] Conventional fashion and beauty recommendation systems often require users to input information online and wait for a long time before receiving recommendations. They are also difficult to use in physical stores, lacking real-time support for users to try on clothes and check makeup looks. Furthermore, they lack the means to provide personalized advice based on a user's skin type or hair concerns. Therefore, there was a need for a system that provides optimal fashion and beauty recommendations in real time in physical stores and supports users in making their choices.

[1175] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1176] In this invention, the server includes means for inputting basic information about a user, means for inputting a face photo and desired beauty style information about the user, means including an artificial intelligence algorithm for analyzing the input information and making optimal fashion, makeup, and hairstyle suggestions, means for providing information on where to purchase the suggested fashion and beauty items and beauty salon information, means for providing appropriate advice and a list of related products based on the user's skin type or hair concerns, and means for receiving real-time fashion and beauty suggestions in a physical store via a smart device. This allows users to receive optimal fashion and beauty suggestions in real time in a physical store, enabling them to efficiently and effectively choose fashion and decide on a beauty style.

[1177] "Basic information" refers to information about a user's personal characteristics and attributes, such as age, sex, height, weight, skin color, and budget.

[1178] A "face photo" is image data of a photograph of a user's face.

[1179] "Beauty style information" is information about the makeup and hairstyle style desired by the user.

[1180] An "artificial intelligence algorithm" is a calculation method that analyzes input information and automatically generates optimal fashion, makeup, and hairstyle suggestions for users.

[1181] "Purchase Information" refers to information about locations or web pages where suggested fashion and beauty items can be purchased.

[1182] "Beauty salon information" refers to information about the location of stores that offer beauty services and how to make reservations.

[1183] "Skin type" refers to the type and characteristics of the user's skin, such as oily skin, dry skin, or sensitive skin.

[1184] "Hair problems" refers to problems or concerns that users have with their hair, such as hair loss, split ends, and dandruff.

[1185] "Appropriate advice" refers to solutions and care methods provided based on the user's skin type and hair concerns.

[1186] "Related Product Listings" refers to a list of products or services related to the relevant advice.

[1187] "Smart devices" refers to portable information devices that can connect to the Internet, such as smartphones and smart glasses.

[1188] "In real time in the physical store" means that suggestions and information are provided instantly on the spot while the customer is in the store.

[1189] To implement this invention, a system is required in which users, terminals, and servers each play a specific role.

[1190] Entering user information

[1191] Users use devices such as smartphones or smart glasses to enter basic information such as age, gender, height, weight, skin tone, and budget, which is collected through applications installed on the devices.

[1192] Register your face photo and beauty style

[1193] Next, the user captures a photo of their face and inputs their desired beauty style information. For example, they can select specific makeup and hairstyles, such as natural makeup or a bob cut. This photo and beauty style information are also collected by the device.

[1194] Sending information

[1195] The collected basic information, face photo, and desired beauty style information are compiled in JSON format and sent to the server, which then receives the user's information and begins analyzing it.

[1196] AI-powered analysis and recommendation generation

[1197] The server analyzes the received user information using an artificial intelligence algorithm. The algorithm used here includes, for example, a generative AI model using deep learning. Based on this analysis, the server generates optimal fashion, makeup, and hairstyle suggestions for the user.

[1198] View Suggestions

[1199] The generated suggestions are sent from the server to the user's device, where they are displayed visually. The user can review the suggestions and, if they like them, click the button to display more information.

[1200] Providing supplier information

[1201] Information on where to purchase the suggested fashion and beauty items and beauty salons is also displayed on the device. Users can review this information and click on the purchase or reservation link to be redirected to the purchase website or salon's reservation system.

[1202] Real-time proposals in physical stores

[1203] In-store use allows users to use smart glasses or smartphones to receive real-time fashion and beauty recommendations, allowing them to try on suggested styles in the fitting room or cosmetics section.

[1204] Hardware and software used

[1205] The hardware used includes smartphones, smart glasses, cameras, etc. The software includes Python, OpenCV, and the requests library, which are combined to input, send, analyze, generate suggestions, and display information.

[1206] Specific examples

[1207] For example, suppose a 35-year-old female user enters her age, gender, height 160cm, weight 55kg, skin color light beige, and budget 20,000 yen. A photo of her face is captured, and her desired style of natural makeup and bob cut is selected and sent to the server. The server analyzes the data and suggests a style of autumn casual fashion, natural makeup, and bob cut. These suggestions are displayed in real time on the user's smart glasses, and the user can try them out in the store.

[1208] Prompt Sentence Examples

[1209] "Enter basic information such as your age, gender, height, weight, skin tone, and budget. Then, capture a photo of your face and select the makeup and hairstyle style you want. The AI ​​will then suggest the best fashion, makeup, and hairstyle for you."

[1210] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1211] Step 1: Enter your user information

[1212] Users use a smartphone or smart glasses to input basic information such as age, gender, height, weight, skin color, and budget. The input information is collected by an application installed on the device. This basic information becomes input data. The input data is packaged in JSON format and sent to the server.

[1213] Step 2: Register your face photo and beauty style

[1214] The user captures a photo of their face using the device's camera and inputs the desired makeup and hairstyle. For example, they can select a style such as natural makeup or a bob cut. This data is also compiled in JSON format and sent to the server. In this step, the face photo and beauty style information become input data and are sent to the server.

[1215] Step 3: Analyze the information and generate recommendations

[1216] The server uses an artificial intelligence algorithm to analyze the received data based on the basic information, face photo, and beauty style information. A generative AI model is used in this process. Specifically, based on the input data, the system extracts data features and performs statistical inference to suggest the most suitable fashion and beauty styles for users of the same age. The output is a recommendation of the most suitable fashion, makeup, and hairstyle for the user.

[1217] Step 4: Submit your proposal

[1218] The server generates and sends the proposed results to the user's device. The proposed results consist of both visual (images and graphics) and text formats. This output data is sent to the user's device and received by the application.

[1219] Step 5: Viewing the Suggestion Results

[1220] The user's device displays the received proposal results on the application's user interface. The user can check the displayed proposals and click a button to display detailed information. The input data for this step are the proposal results sent from the server, and the output data is the content displayed on the user interface.

[1221] Step 6: Provide supplier and beauty salon information

[1222] The user's device displays the suggested fashion and beauty item supplier information and beauty salon information to the user. The user can confirm this information and click on the purchase or reservation link to be redirected to the supplier website or salon reservation system. The input data is the supplier information and beauty salon information included in the recommendation results, and the output data is the redirected web page.

[1223] Step 7: Real-time in-store recommendations

[1224] When a user uses smart glasses or a smartphone in a physical store, the device displays appropriate fashion and beauty suggestions in real time. The user can try on the suggested styles in the store. Input data includes the user's current location in the store and a real-time facial photo, and the output data is the suggestions that are best suited to the current situation.

[1225] Step 8: Recording economic behavior data

[1226] The server records the user's purchasing and salon reservation behavior and stores it in an economic behavior database. This data is used to analyze the user's preferences and behavioral patterns. The input data are the purchasing and reservation actions, and the output data is the updated economic behavior database.

[1227] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1228] The present invention relates to a system that combines a user's basic information, facial photo, desired beauty style information, and an emotion engine that recognizes the user's emotions. This makes it possible to provide optimal fashion, makeup, and hairstyle suggestions while taking the user's emotions into consideration. The specific configuration and operation of the system are described below.

[1229] System configuration and operation

[1230] 1. Enter and submit user information

[1231] The user launches the application and enters basic information such as age, gender, height, weight, skin color, budget, etc. The device packages the entered basic information in JSON format and sends it to the server.

[1232] 2. Register your face photo and enter your beauty style

[1233] The user then uploads a photo of their face and selects the desired makeup and hairstyle, which is also compiled into JSON format by the device and sent to the server.

[1234] 3. Emotion Recognition by Emotion Engine

[1235] The server analyzes the received facial photos and uses an emotion engine to recognize the user's emotions, which are then used to adjust fashion and beauty suggestions.

[1236] 4. Comprehensive production by AI

[1237] The server analyzes the received basic information, emotional information, facial photo, and desired beauty style information, and uses an artificial intelligence algorithm to comprehensively produce the most suitable fashion, makeup, and hairstyle for the user. This information is generated as an integrated visual proposal, which is adjusted based on the emotional information. The proposal results are sent from the server to the device.

[1238] 5. Confirmation of proposal and provision of further information

[1239] The device analyzes the recommendations received from the server and displays visual suggestions on the app's interface. The user can review the suggestions and, if they like them, click the "View More Information" button. This will display information about where to purchase the suggested fashion and beauty items and beauty salons.

[1240] 6. Purchase and reservation assistance

[1241] If the user wishes to purchase the suggested item or make a salon reservation, they operate the device and click on the link, which redirects the user to the purchase website or salon reservation system. The server records this action and stores it in the economic behavior database.

[1242] 7. Providing advice on skin type and hair problems

[1243] Additionally, users can input their skin type and hair concerns. The device sends this information to the server, which uses AI to generate appropriate advice and a list of related products. Emotional information is taken into account to provide tailored advice. The device then displays the received advice and product list to the user.

[1244] Specific examples

[1245] Example 1: Entering basic information and beauty information

[1246] A 35-year-old female user logs into the app and enters her age, gender, height 160cm, weight 55kg, skin color light beige, and budget of 20,000 yen. Next, she uploads a photo of her face and selects her desired style of natural makeup and bob cut. The device compiles this information into JSON format and sends it to the server.

[1247] Example 2: Emotion Recognition with an Emotion Engine

[1248] The server analyzes the received facial photo, and the emotion engine recognizes the user's emotion as "happy." This emotion information is used to tailor the suggestions.

[1249] Example 3: Proposal generation and display

[1250] Based on the input basic information, desired style information, and emotional information, the server generates style suggestions, such as autumn casual fashion, natural makeup, and bob cuts, suitable for women of the same age. The results of these suggestions are sent to the device as visuals. The device displays the suggestions to the user, who can click a button to view more information.

[1251] Example 4: Purchasing and reservation assistance

[1252] The user likes the suggested items, a blouse and a skirt, and clicks on a link to purchase them. The device redirects the user to the website where the items are purchased, and the server records this action.

[1253] In this way, the system based on the present invention provides users with customized fashion and beauty suggestions that take emotional information into consideration, creating an environment where they can make purchases and reservations quickly.

[1254] The processing flow will be explained below.

[1255] Program processing steps

[1256] Step 1: Enter your user information

[1257] Behavior 1.1

[1258] Users launch the app and enter basic information such as age, gender, height, weight, skin color, and budget into a form within the app.

[1259] Behavior 1.2

[1260] The device packages the basic information entered in JSON format and sends it to the server.

[1261] Step 2: Register your face photo and enter your beauty style

[1262] Behavior 2.1

[1263] Users can upload a photo of their face to the app using their device's camera or by selecting a suitable photo from their gallery.

[1264] Behavior 2.2

[1265] The user selects the desired makeup and hairstyle (e.g., natural makeup, bob cut).

[1266] Behavior 2.3

[1267] The device compiles the facial photo and beauty style information into JSON format and sends it to the server.

[1268] Step 3: Emotion recognition by the emotion engine

[1269] Behavior 3.1

[1270] The server analyzes the received facial photo using an emotion engine to recognize the user's emotions.

[1271] Behavior 3.2

[1272] The server stores the recognized emotion information in a database and prepares it for further processing.

[1273] Step 4: Comprehensive production by AI

[1274] Behavior 4.1

[1275] The server launches an artificial intelligence algorithm to analyze basic information, emotional information, facial photos, and desired beauty style information.

[1276] Behavior 4.2

[1277] The server's artificial intelligence algorithm generates optimal fashion, makeup, and hairstyle suggestions based on the original input information and emotional information.

[1278] Behavior 4.3

[1279] The server packages the generated proposal results in JSON format and sends them to the terminal.

[1280] Step 5: Review your proposal and provide further information

[1281] Operation 5.1

[1282] The device analyzes the suggestions received from the server and displays visual suggestions on the app interface.

[1283] Behavior 5.2

[1284] The user reviews the suggestions and, if they like them, clicks the button to display more information.

[1285] Operation 5.3

[1286] The device displays a list of suggested fashion and beauty items, as well as information on where to buy them and beauty salons.

[1287] Step 6: Purchase and reservation assistance

[1288] Operation 6.1

[1289] If the user wishes to purchase the suggested item or make a salon appointment, they click on the displayed link.

[1290] Operation 6.2

[1291] Based on the link clicked, the device redirects the user to the purchase website or beauty salon reservation system.

[1292] Operation 6.3

[1293] The server records the user's purchasing and reservation behavior and stores it in an economic behavior database.

[1294] Step 7: Providing advice on skin type and hair concerns

[1295] Operation 7.1

[1296] Users can also enter their skin type and hair concerns (e.g., dry skin, dandruff, etc.) into the app.

[1297] Operation 7.2

[1298] The device packages the entered information about skin type and hair concerns in JSON format and sends it to the server.

[1299] Operation 7.3

[1300] The server uses artificial intelligence algorithms to generate appropriate advice and a list of related products based on the input skin type and hair concerns, while also taking emotional information into account.

[1301] Operation 7.4

[1302] The server packages the generated advice and product list in JSON format and sends it to the terminal.

[1303] Operation 7.5

[1304] The device displays the received advice and a list of related products on the app interface.

[1305] Specific examples

[1306] Example 1: Entering basic information and beauty information

[1307] A 35-year-old female user logs into the app and enters her age, gender, height 160cm, weight 55kg, skin color light beige, and budget of 20,000 yen. Next, she uploads a photo of her face and selects her desired style of natural makeup and bob cut. The device compiles this information into JSON format and sends it to the server.

[1308] Example 2: Emotion Recognition with an Emotion Engine

[1309] The server analyzes the received facial photo, and the emotion engine recognizes the user's emotion as "happy." This emotion information is used to tailor the suggestions.

[1310] Example 3: Proposal generation and display

[1311] Based on the input basic information, desired style information, and emotional information, the server generates style suggestions, such as autumn casual fashion, natural makeup, and bob cuts, suitable for women of the same age. The results of these suggestions are sent to the device as visuals. The device displays the suggestions to the user, who can click a button to view more information.

[1312] Example 4: Purchasing and reservation assistance

[1313] The user likes the suggested items, a blouse and a skirt, and clicks on a link to purchase them. The device redirects the user to the website where the items are purchased, and the server records this action.

[1314] By following these steps, the system based on the present invention provides users with customized fashion and beauty suggestions that take emotional information into account, creating an environment for quick purchases and reservations.

[1315] Example 2

[1316] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1317] Current beauty and fashion recommendation systems often do not adequately consider users' emotions or detailed personal information. As a result, the recommendations provided may not meet users' expectations. Furthermore, many systems do not include a means to smoothly complete purchase or reservation procedures, resulting in a lack of convenience for users. This poses a challenge, resulting in a decrease in user satisfaction.

[1318] The identification process by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes: means for inputting basic information about the user; means for inputting a facial photo and desired beauty style information about the user; artificial intelligence algorithm means for analyzing the input information and making optimal fashion, makeup, and hairstyle suggestions; emotion recognition means for analyzing the received facial photo and recognizing the user's emotions; means for generating suggestions and adjusting them based on the emotion information; means for providing information about where to purchase the suggested fashion and beauty items and beauty salons; and means for providing appropriate advice and a list of related products based on the user's skin type or hair concerns. This makes it possible to provide more personalized suggestions that take into account the user's emotions and detailed personal information, facilitating smoother purchasing and reservation procedures.

[1319] "Basic information" refers to personal information such as the user's age, gender, height, weight, skin color, and budget.

[1320] "Face Photo" refers to an image of a user's face uploaded by the user.

[1321] "Beauty style information" refers to information about the user's desired beauty style, such as desired makeup and hairstyle.

[1322] "Artificial intelligence algorithm" refers to an algorithm that analyzes input information and generates optimal fashion suggestions, makeup suggestions, and hairstyle suggestions.

[1323] "Emotion recognition means" refers to a system that analyzes the received facial photo and recognizes the user's emotions.

[1324] The "means for adjusting the recommendation result" refers to a means for adjusting the generated recommendation result based on the recognized emotion.

[1325] "Purchaser Information" refers to information about where to purchase suggested fashion and beauty items.

[1326] "Beauty salon information" refers to information about a beauty salon that provides the proposed beauty services.

[1327] "Skin type" refers to the nature and condition of the user's skin.

[1328] "Hair concerns" refers to the hair-related problems and hopes that users have.

[1329] "Advice means" refers to a means for providing appropriate advice based on the user's skin type and hair concerns.

[1330] "Related Product Listings" refers to lists of beauty products or items recommended based on the advice provided.

[1331] "Means for inputting basic information" refers to the interface and functions that allow users to input basic information.

[1332] "Means for inputting face photo and beauty style information" refers to an interface or function that allows a user to input a face photo and desired beauty style information.

[1333] The present invention relates to a system that integrates a user's basic information, facial photo, desired beauty style information, and emotional information to provide optimal fashion, makeup, and hairstyle suggestions. This system can personalize suggestions using an emotion engine that recognizes the user's emotions and an artificial intelligence algorithm. The specific configuration and operation of the present invention are described below.

[1334] composition

[1335] 1. Enter and submit user information

[1336] The user launches the application and enters basic information such as age, gender, height, weight, skin color, and budget. The device packages the entered basic information in JSON format and sends it to the server. The software used is an interface for entering basic information and a program for converting it to JSON format.

[1337] 2. Register your face photo and enter your beauty style

[1338] The user then uploads a photo of their face and selects the desired makeup and hairstyle. This is also compiled into JSON format by the device and sent to the server. The software used is an interface for uploading face photos and selecting beauty styles, as well as a comprehensive data package program.

[1339] 3. Emotion Recognition by Emotion Engine

[1340] The server analyzes the received facial photos and recognizes the user's emotions using an emotion engine. This emotion information is used to adjust fashion and beauty suggestions. The hardware used is a high-performance image analysis server, and the software is an emotion recognition engine (e.g., Google Cloud Vision API).

[1341] 4. Comprehensive production by AI

[1342] The server analyzes the received basic information, emotional information, facial photo, and desired beauty style information, and uses artificial intelligence algorithms to comprehensively produce the most suitable fashion, makeup, and hairstyle for the user. This information is generated as an integrated visual proposal, which is adjusted based on the emotional information. The proposal results are sent from the server to the device. The software used is a machine learning model (e.g., TensorFlow) and an image generation algorithm.

[1343] 5. Confirmation of proposal and provision of further information

[1344] The device analyzes the recommendation results received from the server and displays visual suggestions on the app's interface. The user checks the suggestions and clicks the display button if more information is needed. This displays information on where to purchase the suggested fashion and beauty items and beauty salons. The software used is the recommendation result display interface and the detailed information display program.

[1345] 6. Purchase and reservation assistance

[1346] If the user wishes to purchase the suggested item or make a salon reservation, they operate the device and click on the link. This causes the device to redirect the user to the purchasing website or salon reservation system. The server records this behavior and stores it in an economic behavior database. The software used is a redirect program and a behavior recording program.

[1347] 7. Providing advice on skin type and hair problems

[1348] Additionally, users can enter their skin type and hair concerns. The device sends this information to a server, which uses AI to generate appropriate advice and a list of related products. Emotional information is taken into account to provide tailored advice. The device then displays the received advice and product list to the user. The software used is a skin type and hair concern input interface and a program that generates a list of related products.

[1349] Specific examples

[1350] Example 1: Entering basic information and beauty information

[1351] A 35-year-old female user logs into the app and enters her age, gender, height 160cm, weight 55kg, skin color light beige, and budget of 20,000 yen. Next, she uploads a photo of her face and selects her desired style of natural makeup and bob cut. The device compiles this information into JSON format and sends it to the server.

[1352] Example 2: Emotion Recognition with an Emotion Engine

[1353] The server analyzes the received facial photo, and the emotion engine recognizes the user's emotion as "happy." This emotion information is used to tailor the suggestions.

[1354] Example 3: Proposal generation and display

[1355] Based on the user's basic information, desired style, and emotional information, the server proposes styles suitable for women of the same age, such as autumn casual fashion, natural makeup, and bob cuts. The proposed results are sent to the device as visuals. The device displays the proposed content to the user, who can click a button to view more information.

[1356] Example 4: Purchasing and reservation assistance

[1357] The user likes the suggested items, a blouse and a skirt, and clicks on a link to purchase them. The device redirects the user to the website where the items are purchased, and the server records this action.

[1358] Prompt Sentence Examples

[1359] "Please suggest something that suits the casual fashion and natural makeup worn by a 35-year-old woman. Furthermore, the user is showing the emotion 'happy.'"

[1360] This allows users to get the best fashion and beauty suggestions from the generative AI model.

[1361] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1362] Step 1:

[1363] A user launches the application and logs in. They then enter basic information such as age, gender, height, weight, skin tone, and budget.

[1364] Input: Basic information entered by the user.

[1365] Specific operation: User authentication is performed on the application login screen, and after moving to the main screen, a basic information input form is displayed. When the user clicks the submit button, the input data is converted to JSON format on the terminal side.

[1366] Output: Basic information converted to JSON format.

[1367] Step 2:

[1368] The terminal transmits the converted basic information to the server.

[1369] Input: Basic information in JSON format.

[1370] Specific operation: The device sends JSON data to the server using the REST API.

[1371] Output: Basic information data received by the server.

[1372] Step 3:

[1373] Users upload a photo of their face and select the desired makeup and hairstyle.

[1374] Input: A photo of the user's face and desired beauty style.

[1375] Specific operation: A face photo upload screen is displayed, the user selects a photo, clicks the upload button, and after selecting a beauty style, the device compiles this information into JSON format.

[1376] Output: A JSON-formatted face photo and desired beauty style information.

[1377] Step 4:

[1378] The device sends a facial photo and beauty style information in JSON format to the server.

[1379] Input: Face photo and beauty style information in JSON format.

[1380] Specific operation: The device sends data to the server using the REST API.

[1381] Output: Face photo and beauty style information data received by the server.

[1382] Step 5:

[1383] The server analyzes the received facial photograph and recognizes the user's emotions using an emotion recognition engine.

[1384] Input: Facial photo data.

[1385] Specific operation: The server uses a high-performance image analysis server to run a facial recognition algorithm and analyzes emotions using an emotion recognition engine (e.g., Google Cloud Vision API).

[1386] Output: Emotion information data.

[1387] Step 6:

[1388] The server analyzes basic information, emotional information, facial photos, and desired beauty style information, and uses artificial intelligence algorithms to produce the optimal fashion, makeup, and hairstyle.

[1389] Input: Basic information, emotional information, face photo, desired beauty style information.

[1390] How it works: The server analyzes this information using a data analysis module and generates suggestions using a machine learning model (e.g., TensorFlow), which are then visualized using an image generation algorithm.

[1391] Output: Visual suggestion data.

[1392] Step 7:

[1393] The server transmits the visual suggestion data to the terminal.

[1394] Input: Visual suggestion data.

[1395] Specific operation: The server sends visual suggestion data to the device using the REST API.

[1396] Output: Visual suggestion data received by the device.

[1397] Step 8:

[1398] The device analyzes the received suggestions and displays visual suggestions on the app interface. The user can review the suggestions and click the view button to view more information.

[1399] Input: Visual suggestion data.

[1400] What happens: The device parses the received content and displays it in the app's UI as images and text. When the user clicks the details button, the additional information expands to show more information.

[1401] Output: Visual suggestions and detailed information displayed in the app's UI.

[1402] Step 9:

[1403] If the user wishes to purchase the suggested item or make a salon reservation, he or she operates the terminal and clicks on the link.

[1404] Input: Purchase or reservation link.

[1405] What happens: When a user clicks on a purchase link, their device's browser opens to the website where the purchase is made.

[1406] Output: Purchasing website or salon booking system.

[1407] Step 10:

[1408] The server records user behavior and stores it in an economic behavior database.

[1409] Input: User behavior data.

[1410] Specific operation: The server logs users' link clicks and purchase behavior and stores them in an economic behavior database.

[1411] Output: Stored behavioral data.

[1412] Step 11:

[1413] The user enters additional information such as their skin type and hair concerns, and the device sends this to the server.

[1414] Input: Skin type and hair concerns.

[1415] Specific behavior: When the user fills out the additional information input form and clicks the submit button, the device converts the data into JSON format and sends it to the server.

[1416] Output: Skin type and hair problem information received by the server.

[1417] Step 12:

[1418] The server uses AI to generate appropriate advice and a list of related products based on skin type and hair concerns.

[1419] Input: Skin type and hair concerns.

[1420] How it works: The server uses an AI model to analyze the input information and generate advice and product lists.

[1421] Output: Advice and related product list.

[1422] Step 13:

[1423] The terminal displays the received advice and related product list to the user.

[1424] Input: Advice and related product listings.

[1425] Specific behavior: The device parses the received content and displays it in the app's UI.

[1426] Output: Advice and related product list displayed in the app UI.

[1427] (Application example 2)

[1428] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1429] Today's consumers demand personalized fashion and beauty recommendations, but existing systems often only offer one-size-fits-all recommendations, resulting in low satisfaction. Furthermore, the effectiveness of recommendations is limited because they are not tailored based on real-time emotional information. Furthermore, there is a lack of support for in-store purchases and try-ons, creating a need for ways to improve the user shopping experience.

[1430] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes: means for inputting basic information about the user; means for inputting a facial photo and desired beauty style information about the user; means including an artificial intelligence algorithm for analyzing the input information and making optimal fashion, makeup, and hairstyle suggestions; means for providing information on where to purchase the suggested fashion and beauty items and beauty salon information; means for providing appropriate advice and a list of related products based on the user's skin type or hair concerns; means for displaying the suggestion results on smart glasses worn by the user in a physical store, recognizing emotions in real time, and adjusting the suggestion content based on the emotional information; and means for supporting the user in trying on and purchasing items they like through the smart glasses. This enables the provision of personalized suggestions and adjustments based on real-time emotional information, as well as an improved shopping experience in a physical store.

[1431] "Means for inputting basic user information" refers to devices or applications for inputting basic information such as the user's age, gender, height, weight, skin color, and budget.

[1432] The "means for inputting the user's facial photograph and desired beauty style information" refers to a device or application for inputting the user's facial photograph and beauty style information such as desired makeup and hairstyle.

[1433] "Means including an artificial intelligence algorithm for analyzing input information and making optimal suggestions for fashion, makeup, and hairstyle" refers to systems or software that include an artificial intelligence algorithm used to analyze input user information and make suggestions for optimal fashion, makeup, and hairstyle.

[1434] The "means for providing information on where to purchase suggested fashion and beauty items and information on beauty salons" refers to a system or application for providing information on where to purchase suggested fashion and beauty items and information on beauty salons.

[1435] "Means for providing appropriate advice and a list of related products based on the user's skin type or hair concerns" refers to a system or software for providing appropriate advice and a list of related products based on the user's skin type or hair concerns.

[1436] "A means for displaying the proposal results on smart glasses worn by the user in a physical store, recognizing emotions in real time, and adjusting the proposal content based on that emotional information" refers to a system or algorithm for displaying the proposal results on smart glasses worn by the user in a physical store, recognizing the user's emotions in real time, and adjusting the proposal content based on that emotional information.

[1437] "Means for supporting users in trying on and purchasing items they like through smart glasses" refers to a system or software that supports users in trying on and purchasing items they like through the UI of smart glasses.

[1438] The present invention relates to a system that combines a user's basic information, facial photo, desired beauty style information, and an emotion engine that recognizes the user's emotions. This makes it possible to provide optimal fashion, makeup, and hairstyle suggestions while taking the user's emotions into consideration. The specific configuration and operation of the system are described below.

[1439] System configuration and operation

[1440] 1. Enter and submit user information

[1441] Users wear smart glasses and input basic information such as age, gender, height, weight, skin color, and budget. The smart glasses package the input basic information in JSON format and send it to the server. The smart glasses are equipped with a camera and touchpad, making it easy for users to input information.

[1442] 2. Register your face photo and enter your beauty style

[1443] The user then takes a photo of their face using the smart glasses' camera and selects the desired makeup and hairstyle, which is then compiled into JSON format by the smart glasses and sent to the server.

[1444] 3. Emotion Recognition by Emotion Engine

[1445] The server analyzes the received facial photo and uses an emotion engine (emotion recognition technology such as DeepFace) to recognize the user's emotions. This emotional information is used to adjust fashion and beauty suggestions.

[1446] 4. Comprehensive production by AI

[1447] The server analyzes the received basic information, emotional information, facial photo, and desired beauty style information, and then uses an artificial intelligence algorithm (e.g., a model using Keras) to comprehensively produce the most suitable fashion, makeup, and hairstyle for the user. This information is then generated as an integrated visual suggestion, which is adjusted based on the emotional information. The suggestion results are sent from the server to the smart glasses.

[1448] 5. Confirmation of proposal and provision of further information

[1449] The smart glasses analyze the recommendations received from the server and display visual suggestions to the user. The user reviews the recommendations and, if they like them, clicks the display details button on the touchpad. This displays information on where to purchase the suggested fashion and beauty items and beauty salons on the smart glasses' display.

[1450] 6. Purchase and reservation assistance

[1451] If the user chooses a suggested item and wishes to purchase or try it on, they can operate the smart glasses' interface and click the corresponding button, which will redirect the user to the purchase website or salon's reservation system. The server records this action and stores it in the economic behavior database.

[1452] 7. Providing advice on skin type and hair problems

[1453] Additionally, users can input their skin type and hair concerns through the smart glasses. The smart glasses then send this information to the server, which uses AI to generate appropriate advice and a list of related products. Emotional information is taken into account to provide tailored advice. The smart glasses then display the received advice and product list to the user.

[1454] Specific examples

[1455] Example 1: Entering basic information and beauty information

[1456] A 30-year-old female user puts on smart glasses and enters her age, gender, height 160cm, weight 55kg, skin color light beige, and budget of 20,000 yen. Next, she takes a photo of her face with the smart glasses' camera and selects her desired style of natural makeup and short hair. The smart glasses then compile this information into JSON format and send it to the server.

[1457] Example 2: Emotion Recognition with an Emotion Engine

[1458] The server analyzes the received facial photo and uses an emotion engine (DeepFace) to recognize the user's emotion as "happy." This emotion information is used to adjust the suggestions.

[1459] Example 3: Proposal generation and display

[1460] Based on the input basic information, desired style information, and emotional information, the server generates style suggestions, such as autumn casual fashion, natural makeup, and short haircuts, suitable for women of the same age. The results of these suggestions are sent as visuals to the smart glasses. The smart glasses then display the suggestions to the user, who can then operate the touchpad to view more information.

[1461] Example 4: Purchasing and reservation assistance

[1462] The user likes the suggested items, a jacket and pants, and clicks on a link to purchase them. The smart glasses redirect the user to the website where they purchased the items, and the server records this action.

[1463] Prompt Sentence Examples

[1464] Recognize user emotions and provide customized fashion and beauty recommendations based on the following information:

[1465] Age: 30

[1466] Gender: Female

[1467] Height: 160cm

[1468] Weight: 55kg

[1469] Skin Tone: Light Beige

[1470] Budget: 20,000 yen

[1471] Face photo: face.jpg

[1472] Suggest the best style based on user sentiment.

[1473] In this way, the system based on the present invention provides users with customized fashion and beauty suggestions that take emotional information into account, allowing for real-time adjustments and quick purchase and try-on in-store.

[1474] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1475] Step 1:

[1476] The user puts on the smart glasses and inputs basic information, such as age, gender, height, weight, skin tone, and budget, using the touchpad on the smart glasses interface. The input basic information is converted into JSON format and sent to the server, where the user information is stored and available for subsequent analysis.

[1477] Step 2:

[1478] The user takes a photo of their face using the smart glasses' camera and inputs their desired beauty style information (e.g., type of makeup, hairstyle). The smart glasses also compile this information into JSON format and send it to the server. The server then receives the user's face photo and desired style information, and is ready for emotion recognition.

[1479] Step 3:

[1480] The server analyzes the received facial photo and recognizes the user's emotions using an emotion engine (such as DeepFace). Specifically, it uses the DeepFace library to extract facial features from the facial photo and determine the emotion based on those features. The emotional information obtained here is then used to adjust the fashion suggestions.

[1481] Step 4:

[1482] The server combines the user's basic information, face photo, desired beauty style information, and emotional information received from the user and analyzes it using an artificial intelligence algorithm (e.g., a generative AI model using Keras). This analysis generates optimal fashion, makeup, and hairstyle recommendations. The AI ​​converts the user's information into personalized recommendations based on statistical data and trend information.

[1483] Step 5:

[1484] The server sends the generated suggestions to the smart glasses worn by the user in the physical store. The smart glasses display the suggestions to the user. The user can review the suggestions using the touchpad, and if they like them, they can access more detailed information by tapping the display button for more information.

[1485] Step 6:

[1486] If the user likes any of the suggested fashion items, they will be assisted in trying them on and purchasing them through the smart glasses' UI. Specifically, the smart glasses will provide fitting room reservations and detailed product information based on the user's preferences, and guide the user through the operations required for trying on and purchasing. This allows the user to quickly try on and purchase items using the smart glasses.

[1487] Step 7:

[1488] The server records the user's economic behavior data and stores it in a database. Specifically, it collects and records behavioral data such as which offers the user accepted, which items they purchased, and which beauty salons they booked. This data can be used for subsequent analysis and to improve the accuracy of future offers.

[1489] Step 8:

[1490] Users also input their skin type and hair concerns through the smart glasses. The smart glasses send this information to a server, which uses AI to generate appropriate advice and a list of related products. The advice takes emotional information into account, and is received by the smart glasses and displayed to the user.

[1491] Through these steps, users can receive personalized fashion and beauty recommendations based on their emotions, with real-time adjustments and a quick in-store purchasing experience.

[1492] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1493] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1494] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[1495] [Fourth embodiment]

[1496] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[1497] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1498] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1499] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[1500] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1501] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1502] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1503] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[1504] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1505] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1506] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1507] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1508] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1509] The present invention relates to a system that utilizes artificial intelligence to provide a means for users to input their basic information, a photograph of their face, and information about their desired beauty style, and then analyzes this information to provide comprehensive fashion, makeup, and hairstyle suggestions. This system automatically provides optimal fashion and beauty suggestions by having users input the necessary information.

[1510] System configuration and operation

[1511] 1. Enter and submit user information

[1512] The user launches the application and enters basic information such as age, gender, height, weight, skin tone, budget, etc. The device packages the entered information in JSON format and sends it to the server.

[1513] 2. Register your face photo and beauty style

[1514] The user then uploads a photo of their face and selects the desired makeup and hairstyle, which is also compiled into JSON format by the device and sent to the server.

[1515] 3. AI-generated recommendations

[1516] Based on the received information, the server uses an artificial intelligence algorithm to comprehensively produce the most suitable fashion, makeup, and hairstyle for the user. This information is analyzed and an integrated visual proposal is generated. The proposal results are then sent from the server to the device.

[1517] 4. Review the proposal and view more information

[1518] The device receives the recommendations from the server and displays them to the user on the app interface. The user can review the recommendations and, if they like them, click the "Display details" button. This will display information about where to purchase the recommended fashion and beauty items and beauty salons.

[1519] 5. Purchase and reservation assistance

[1520] If the user wishes to purchase the suggested item or make a salon reservation, they operate the device and click on the link, which redirects the user to the purchase website or salon reservation system. The server records this action and stores it in the economic behavior database.

[1521] 6. Providing advice on skin type and hair problems

[1522] Additionally, users can input their skin type and hair concerns. The device sends this information to the server, which uses AI to generate appropriate advice and a list of related products and send them to the device. The device then displays the received advice and product list to the user.

[1523] Specific examples

[1524] Example 1: Entering basic information and beauty information

[1525] A 35-year-old female user logs into the app and enters her age, gender, height 160cm, weight 55kg, skin color light beige, and budget of 20,000 yen. Next, she uploads a photo of her face and selects her desired style of natural makeup and bob cut. The device compiles this information into JSON format and sends it to the server.

[1526] Example 2: Proposal generation and display

[1527] The server analyzes the input information and generates style suggestions, such as autumn casual fashion, natural makeup, and bob cuts, suitable for women of the same age. These suggestions are sent as visuals to the device. The device displays the suggestions to the user, who can click a button to view more information.

[1528] Example 3: Purchasing and reservation assistance

[1529] The user likes the suggested items, a blouse and a skirt, and clicks on a link to purchase them. The device redirects the user to the website where the items are purchased, and the server records this action.

[1530] In this way, the system based on the present invention provides users with quick and accurate comprehensive fashion and beauty suggestions, and smoothly links these to purchases and service use.

[1531] The processing flow will be explained below.

[1532] Program processing steps

[1533] Step 1: Enter your user information

[1534] Behavior 1.1

[1535] Users launch the app and enter basic information such as age, gender, height, weight, skin color, and budget into a form within the app.

[1536] Behavior 1.2

[1537] The device packages the basic information entered in JSON format and sends it to the server.

[1538] Step 2: Register your face photo and enter your beauty style

[1539] Behavior 2.1

[1540] Users can upload a photo of their face to the app using their device's camera or by selecting a suitable photo from their gallery.

[1541] Behavior 2.2

[1542] The user selects the desired makeup and hairstyle (e.g., natural makeup, bob cut).

[1543] Behavior 2.3

[1544] The device compiles the facial photo and beauty style information into JSON format and sends it to the server.

[1545] Step 3: Comprehensive production by AI

[1546] Behavior 3.1

[1547] The server then calls an artificial intelligence algorithm to analyze the received basic information, facial photo, and desired beauty style information.

[1548] Behavior 3.2

[1549] The server's artificial intelligence algorithm generates optimal fashion, makeup, and hairstyle suggestions based on the user's age, gender, height, weight, skin color, budget, and desired style information.

[1550] Behavior 3.3

[1551] The server packages the generated proposal results in JSON format and sends them to the terminal.

[1552] Step 4: Review your proposal and provide further information

[1553] Behavior 4.1

[1554] The device analyzes the suggestions received from the server and displays visual suggestions on the app interface.

[1555] Behavior 4.2

[1556] The user reviews the suggestions and, if they like them, clicks the button to display more information.

[1557] Behavior 4.3

[1558] The device displays a list of suggested fashion and beauty items, as well as information on where to purchase them and beauty salons.

[1559] Step 5: Purchase and reservation assistance

[1560] Operation 5.1

[1561] If the user wishes to purchase the suggested item or make a salon appointment, they click on the displayed link.

[1562] Behavior 5.2

[1563] Based on the link clicked, the device redirects the user to the purchase website or beauty salon reservation system.

[1564] Operation 5.3

[1565] The server records the user's purchasing and reservation behavior and stores it in an economic behavior database.

[1566] Step 6: Providing advice on skin type and hair concerns

[1567] Operation 6.1

[1568] Users can also enter their skin type and hair concerns (e.g., dry skin, dandruff, etc.) into the app.

[1569] Operation 6.2

[1570] The device packages the entered information about skin type and hair concerns in JSON format and sends it to the server.

[1571] Operation 6.3

[1572] The server uses artificial intelligence algorithms to generate appropriate advice and a list of related products based on the input skin type and hair concerns.

[1573] Operation 6.4

[1574] The server packages the generated advice and product list in JSON format and sends it to the terminal.

[1575] Operation 6.5

[1576] The device displays the received advice and a list of related products on the app interface.

[1577] ---

[1578] Through these steps, users can receive customized fashion and beauty suggestions through a consistent process, and can quickly make purchases or reservations.

[1579] Example 1

[1580] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1581] Conventional beauty and fashion recommendation systems have the problem that it takes a lot of time and effort for users to match their own information and preferences in detail. In addition, there are few systems that integrate various information to make comprehensive recommendations, and the purchasing and reservation process after the recommendations is often fragmented. This has led to issues such as low user satisfaction and a lack of improvement in system usage efficiency.

[1582] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1583] In this invention, the server includes means for inputting the user's personal information, means for inputting the user's face photo and desired beauty style information, means for packaging the input information into a data format and sending it to the server, means for analyzing the input information using an artificial intelligence algorithm and making optimal fashion, makeup, and hairstyle suggestions, means for displaying the suggestions received from the server to the user, means for providing information on where to purchase the suggested fashion and beauty items and information on beauty salons, and means for providing appropriate advice and a list of related products based on the user's skin type or hair concerns. This allows users to input their own information within a single system, receive optimal fashion and beauty suggestions based on that information, and also enable them to purchase related items and make beauty salon reservations in an integrated manner.

[1584] "User" means an individual who uses the System.

[1585] "Personal information" refers to information such as age, gender, height, weight, skin color, and budget that users enter into the system.

[1586] "Facial photo" refers to image data of a user's face that the user uploads to the system.

[1587] "Beauty style information" refers to information about the makeup and hairstyle style desired by the user.

[1588] A "data format" refers to a format that structures information to make it easier to store and communicate; a commonly used example is the JSON format.

[1589] "Server" refers to the central computer of the system, a computing resource that receives user input, analyzes it using artificial intelligence algorithms, and makes suggestions.

[1590] "Artificial intelligence algorithm" refers to machine learning and deep learning models and methods used to analyze input data and generate optimal fashion, makeup, and hairstyle suggestions.

[1591] "Suggestion results" is a general term for fashion suggestions, makeup suggestions, and hairstyle suggestions that are analyzed by an artificial intelligence algorithm and provided to users.

[1592] "Purchaser Information" refers to information about stores and online stores where suggested fashion and beauty items can be purchased.

[1593] "Beauty salon information" refers to information such as the location and contact details of the beauty salon that offers the proposed beauty style.

[1594] "Skin type" refers to the condition or characteristics of the user's skin (e.g., dry skin, oily skin).

[1595] "Hair concerns" refers to problems with the user's hair or areas they would like to improve (e.g., thinning hair, split ends).

[1596] The "List of Related Products" refers to a list of beauty products recommended based on the user's skin type and hair concerns.

[1597] "Purchase Intent" refers to a state in which a user has indicated an intention to purchase or use a proposed item or service.

[1598] "Redirecting" refers to the act of sending a user to another web page or service.

[1599] "Economic behavior data" refers to behavioral data when users engage in economic activities such as purchases and reservations.

[1600] "Recording and storing" refers to keeping specific information in a database or storage so that it can be accessed later.

[1601] The present invention relates to a system that utilizes artificial intelligence to input a user's basic information, a facial photo, and desired beauty style information, and then analyzes the information to provide comprehensive fashion, makeup, and hairstyle suggestions. This system automatically provides optimal fashion and beauty suggestions by having the user input the necessary information. Specific embodiments are described in detail below.

[1602] Enter and submit user information

[1603] Users launch a dedicated application and enter their personal information, including age, gender, height, weight, skin color, and budget. This information is packaged in JSON format by the device and sent to the server. Users also upload a photo of their face and select their desired beauty style (e.g., natural makeup, bob cut). This information is also packaged in JSON format by the device and sent to the server.

[1604] AI-powered recommendation generation

[1605] The server analyzes the received user information using an artificial intelligence algorithm. This algorithm uses machine learning models such as TensorFlow and PyTorch. The server generates optimal fashion, makeup, and hairstyle suggestions for the user, renders them in a visual format, and sends them to the device. The analyzed and generated suggestions are then sent to the device as JSON-formatted data.

[1606] Displaying the proposed results and providing detailed information

[1607] The device visually displays the received suggestions on the dedicated application interface. Users can check the suggestions displayed and click the confirmation button for more information, which will display information on where to purchase the suggested fashion items and detailed information on beauty salons.

[1608] Purchase and reservation assistance

[1609] If the user wishes to purchase the suggested item or make a reservation at a beauty salon, they click on the provided link on their device. By clicking on the link, the device redirects the user to the product purchase site or beauty salon reservation system. The user's clicking behavior is recorded by the server and later stored in the economic behavior database.

[1610] Advice on skin and hair problems

[1611] Additionally, users can enter additional information about their skin type and hair concerns. The device sends this information to the server, which uses AI algorithms to generate a list of appropriate advice and related products. This list and advice is then sent to the device and displayed to the user.

[1612] Examples and prompts

[1613] Example 1: Entering basic information and beauty information

[1614] A 35-year-old female user logs into the app and enters her age, height 160cm, weight 55kg, skin color light beige, and budget of 20,000 yen. Next, she uploads a photo of her face and selects her desired style of natural makeup and bob cut. The device compiles this information into JSON format and sends it to the server.

[1615] Example 2: Proposal generation and display

[1616] The server analyzes the input information and generates recommendations for autumn casual fashion, natural makeup, and a bob cut that are ideal for a 35-year-old woman, and sends them to the device. The device displays the recommendations and provides more information when the user clicks the "More Info" button.

[1617] Prompt Sentence Examples

[1618] "35-year-old female, height 160cm, weight 55kg, skin tone light beige, budget 20,000 yen. Prefer natural makeup and a bob cut. Please provide suggestions for casual fall fashion and specific shopping information."

[1619] The above is a detailed description of the embodiment of the present invention. This system allows users to efficiently and easily receive suggestions for optimal fashion and beauty styles, and then smoothly take the next action, such as purchasing a product or making a reservation at a beauty salon, based on that information.

[1620] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1621] Step 1: Enter and submit user information

[1622] The user starts the application and enters personal information (age, sex, height, weight, skin color, budget, etc.) by typing this information into a dedicated form.

[1623] Input: Personal information entered by the user

[1624] The terminal packages the entered information into JSON format.

[1625] Data processing: Validate the information entered by the user and convert it to JSON format.

[1626] Output: User details in JSON format

[1627] The terminal sends the generated JSON data to the server via an HTTP POST request.

[1628] Step 2: Register your face photo and beauty style

[1629] Users upload a photo of their face and select the desired beauty style (e.g., natural makeup, bob cut). Specifically, they take a photo of their face using the camera function on their smartphone and upload it to the app.

[1630] Input: Uploaded face photo and selected beauty style information

[1631] The device then reassembles this information into JSON format.

[1632] Data processing: Convert the face photo into the appropriate image format (e.g., JPEG) and include beauty style information in the JSON data.

[1633] Output: JSON formatted face photo and beauty style information

[1634] The terminal sends the generated JSON data to the server via an HTTP POST request.

[1635] Step 3: Generate recommendations with AI

[1636] The server analyzes the received JSON-formatted user information, face photo, and beauty style information using an artificial intelligence model (e.g., TensorFlow, PyTorch).

[1637] Input: JSON format user information, face photo, beauty style information

[1638] The server inputs this information into a generative AI model to generate optimal fashion, makeup, and hairstyle suggestions.

[1639] Data calculation: Generative AI models analyze input information and generate optimal recommendations

[1640] Output: JSON format suggestion results (fashion suggestions, makeup suggestions, hairstyle suggestions)

[1641] The server renders the generated proposal results in a visual format and sends them to the terminal as JSON data.

[1642] Step 4: View the results of the proposal and provide more information

[1643] The device displays the received proposal results on the interface of a dedicated app. The proposals are visually easy to understand.

[1644] Input: JSON formatted proposal results

[1645] Output: Visual suggestion results shown to the user

[1646] The user checks the suggested content and clicks the "Display detailed information" button, which displays information about where to purchase the suggested fashion items and detailed information about beauty salons.

[1647] Step 5: Assist with purchases and reservations

[1648] If users want to purchase or book a featured item or beauty service, they click on a link within the app.

[1649] Input: User clicks

[1650] The terminal responds to the click and redirects the user to a purchasing website or a beauty salon reservation system.

[1651] Data calculation: Link click event and redirect handling

[1652] Output: The redirected web page

[1653] The server records the user's clicking behavior and stores it in an economic behavior database.

[1654] Step 6: Providing advice on skin type and hair concerns

[1655] Users can also enter their skin type and hair concerns (e.g., dry skin, thinning hair).

[1656] Input: Information about your skin type and hair concerns

[1657] The device compiles this information into JSON format and sends it to the server.

[1658] The server passes the received information to an AI algorithm to generate optimal advice and related product lists.

[1659] Data arithmetic: AI algorithms analyze information and generate advice

[1660] Output: JSON formatted advice and related product list

[1661] The terminal displays the received information again to the user, allowing the user to check appropriate advice and related products.

[1662] (Application example 1)

[1663] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1664] Conventional fashion and beauty recommendation systems often require users to input information online and wait for a long time before receiving recommendations. They are also difficult to use in physical stores, lacking real-time support for users to try on clothes and check makeup looks. Furthermore, they lack the means to provide personalized advice based on a user's skin type or hair concerns. Therefore, there was a need for a system that provides optimal fashion and beauty recommendations in real time in physical stores and supports users in making their choices.

[1665] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1666] In this invention, the server includes means for inputting basic information about a user, means for inputting a face photo and desired beauty style information about the user, means including an artificial intelligence algorithm for analyzing the input information and making optimal fashion, makeup, and hairstyle suggestions, means for providing information on where to purchase the suggested fashion and beauty items and beauty salon information, means for providing appropriate advice and a list of related products based on the user's skin type or hair concerns, and means for receiving real-time fashion and beauty suggestions in a physical store via a smart device. This allows users to receive optimal fashion and beauty suggestions in real time in a physical store, enabling them to efficiently and effectively choose fashion and decide on a beauty style.

[1667] "Basic information" refers to information about a user's personal characteristics and attributes, such as age, sex, height, weight, skin color, and budget.

[1668] A "face photo" is image data of a photograph of a user's face.

[1669] "Beauty style information" is information about the makeup and hairstyle style desired by the user.

[1670] An "artificial intelligence algorithm" is a calculation method that analyzes input information and automatically generates optimal fashion, makeup, and hairstyle suggestions for users.

[1671] "Purchase Information" refers to information about locations or web pages where suggested fashion and beauty items can be purchased.

[1672] "Beauty salon information" refers to information about the location of stores that offer beauty services and how to make reservations.

[1673] "Skin type" refers to the type and characteristics of the user's skin, such as oily skin, dry skin, or sensitive skin.

[1674] "Hair problems" refers to problems or concerns that users have with their hair, such as hair loss, split ends, and dandruff.

[1675] "Appropriate advice" refers to solutions and care methods provided based on the user's skin type and hair concerns.

[1676] "Related Product Listings" refers to a list of products or services related to the relevant advice.

[1677] "Smart devices" refers to portable information devices that can connect to the Internet, such as smartphones and smart glasses.

[1678] "In real time in the physical store" means that suggestions and information are provided instantly on the spot while the customer is in the store.

[1679] To implement this invention, a system is required in which users, terminals, and servers each play a specific role.

[1680] Entering user information

[1681] Users use devices such as smartphones or smart glasses to enter basic information such as age, gender, height, weight, skin tone, and budget, which is collected through applications installed on the devices.

[1682] Register your face photo and beauty style

[1683] Next, the user captures a photo of their face and inputs their desired beauty style information. For example, they can select specific makeup and hairstyles, such as natural makeup or a bob cut. This photo and beauty style information are also collected by the device.

[1684] Sending information

[1685] The collected basic information, face photo, and desired beauty style information are compiled in JSON format and sent to the server, which then receives the user's information and begins analyzing it.

[1686] AI-powered analysis and recommendation generation

[1687] The server analyzes the received user information using an artificial intelligence algorithm. The algorithm used here includes, for example, a generative AI model using deep learning. Based on this analysis, the server generates optimal fashion, makeup, and hairstyle suggestions for the user.

[1688] View Suggestions

[1689] The generated suggestions are sent from the server to the user's device, where they are displayed visually. The user can review the suggestions and, if they like them, click the button to display more information.

[1690] Providing supplier information

[1691] Information on where to purchase the suggested fashion and beauty items and beauty salons is also displayed on the device. Users can review this information and click on the purchase or reservation link to be redirected to the purchase website or salon's reservation system.

[1692] Real-time proposals in physical stores

[1693] In-store use allows users to use smart glasses or smartphones to receive real-time fashion and beauty recommendations, allowing them to try on suggested styles in the fitting room or cosmetics section.

[1694] Hardware and software used

[1695] The hardware used includes smartphones, smart glasses, cameras, etc. The software includes Python, OpenCV, and the requests library, which are combined to input, send, analyze, generate suggestions, and display information.

[1696] Specific examples

[1697] For example, suppose a 35-year-old female user enters her age, gender, height 160cm, weight 55kg, skin color light beige, and budget 20,000 yen. A photo of her face is captured, and her desired style of natural makeup and bob cut is selected and sent to the server. The server analyzes the data and suggests a style of autumn casual fashion, natural makeup, and bob cut. These suggestions are displayed in real time on the user's smart glasses, and the user can try them out in the store.

[1698] Prompt Sentence Examples

[1699] "Enter basic information such as your age, gender, height, weight, skin tone, and budget. Then, capture a photo of your face and select the makeup and hairstyle style you want. The AI ​​will then suggest the best fashion, makeup, and hairstyle for you."

[1700] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1701] Step 1: Enter your user information

[1702] Users use a smartphone or smart glasses to input basic information such as age, gender, height, weight, skin color, and budget. The input information is collected by an application installed on the device. This basic information becomes input data. The input data is packaged in JSON format and sent to the server.

[1703] Step 2: Register your face photo and beauty style

[1704] The user captures a photo of their face using the device's camera and inputs the desired makeup and hairstyle. For example, they can select a style such as natural makeup or a bob cut. This data is also compiled in JSON format and sent to the server. In this step, the face photo and beauty style information become input data and are sent to the server.

[1705] Step 3: Analyze the information and generate recommendations

[1706] The server uses an artificial intelligence algorithm to analyze the received data based on the basic information, face photo, and beauty style information. A generative AI model is used in this process. Specifically, based on the input data, the system extracts data features and performs statistical inference to suggest the most suitable fashion and beauty styles for users of the same age. The output is a recommendation of the most suitable fashion, makeup, and hairstyle for the user.

[1707] Step 4: Submit your proposal

[1708] The server generates and sends the proposed results to the user's device. The proposed results consist of both visual (images and graphics) and text formats. This output data is sent to the user's device and received by the application.

[1709] Step 5: Viewing the Suggestion Results

[1710] The user's device displays the received proposal results on the application's user interface. The user can check the displayed proposals and click a button to display detailed information. The input data for this step are the proposal results sent from the server, and the output data is the content displayed on the user interface.

[1711] Step 6: Provide supplier and beauty salon information

[1712] The user's device displays the suggested fashion and beauty item supplier information and beauty salon information to the user. The user can confirm this information and click on the purchase or reservation link to be redirected to the supplier website or salon reservation system. The input data is the supplier information and beauty salon information included in the recommendation results, and the output data is the redirected web page.

[1713] Step 7: Real-time in-store recommendations

[1714] When a user uses smart glasses or a smartphone in a physical store, the device displays appropriate fashion and beauty suggestions in real time. The user can try on the suggested styles in the store. Input data includes the user's current location in the store and a real-time facial photo, and the output data is the suggestions that are best suited to the current situation.

[1715] Step 8: Recording economic behavior data

[1716] The server records the user's purchasing and salon reservation behavior and stores it in an economic behavior database. This data is used to analyze the user's preferences and behavioral patterns. The input data are the purchasing and reservation actions, and the output data is the updated economic behavior database.

[1717] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1718] The present invention relates to a system that combines a user's basic information, facial photo, desired beauty style information, and an emotion engine that recognizes the user's emotions. This makes it possible to provide optimal fashion, makeup, and hairstyle suggestions while taking the user's emotions into consideration. The specific configuration and operation of the system are described below.

[1719] System configuration and operation

[1720] 1. Enter and submit user information

[1721] The user launches the application and enters basic information such as age, gender, height, weight, skin color, budget, etc. The device packages the entered basic information in JSON format and sends it to the server.

[1722] 2. Register your face photo and enter your beauty style

[1723] The user then uploads a photo of their face and selects the desired makeup and hairstyle, which is also compiled into JSON format by the device and sent to the server.

[1724] 3. Emotion Recognition by Emotion Engine

[1725] The server analyzes the received facial photos and uses an emotion engine to recognize the user's emotions, which are then used to adjust fashion and beauty suggestions.

[1726] 4. Comprehensive production by AI

[1727] The server analyzes the received basic information, emotional information, facial photo, and desired beauty style information, and uses an artificial intelligence algorithm to comprehensively produce the most suitable fashion, makeup, and hairstyle for the user. This information is generated as an integrated visual proposal, which is adjusted based on the emotional information. The proposal results are sent from the server to the device.

[1728] 5. Confirmation of proposal and provision of further information

[1729] The device analyzes the recommendations received from the server and displays visual suggestions on the app's interface. The user can review the suggestions and, if they like them, click the "View More Information" button. This will display information about where to purchase the suggested fashion and beauty items and beauty salons.

[1730] 6. Purchase and reservation assistance

[1731] If the user wishes to purchase the suggested item or make a salon reservation, they operate the device and click on the link, which redirects the user to the purchase website or salon reservation system. The server records this action and stores it in the economic behavior database.

[1732] 7. Providing advice on skin type and hair problems

[1733] Additionally, users can input their skin type and hair concerns. The device sends this information to the server, which uses AI to generate appropriate advice and a list of related products. Emotional information is taken into account to provide tailored advice. The device then displays the received advice and product list to the user.

[1734] Specific examples

[1735] Example 1: Entering basic information and beauty information

[1736] A 35-year-old female user logs into the app and enters her age, gender, height 160cm, weight 55kg, skin color light beige, and budget of 20,000 yen. Next, she uploads a photo of her face and selects her desired style of natural makeup and bob cut. The device compiles this information into JSON format and sends it to the server.

[1737] Example 2: Emotion Recognition with an Emotion Engine

[1738] The server analyzes the received facial photo, and the emotion engine recognizes the user's emotion as "happy." This emotion information is used to tailor the suggestions.

[1739] Example 3: Proposal generation and display

[1740] Based on the input basic information, desired style information, and emotional information, the server generates style suggestions, such as autumn casual fashion, natural makeup, and bob cuts, suitable for women of the same age. The results of these suggestions are sent to the device as visuals. The device displays the suggestions to the user, who can click a button to view more information.

[1741] Example 4: Purchasing and reservation assistance

[1742] The user likes the suggested items, a blouse and a skirt, and clicks on a link to purchase them. The device redirects the user to the website where the items are purchased, and the server records this action.

[1743] In this way, the system based on the present invention provides users with customized fashion and beauty suggestions that take emotional information into consideration, creating an environment where they can make purchases and reservations quickly.

[1744] The processing flow will be explained below.

[1745] Program processing steps

[1746] Step 1: Enter your user information

[1747] Behavior 1.1

[1748] Users launch the app and enter basic information such as age, gender, height, weight, skin color, and budget into a form within the app.

[1749] Behavior 1.2

[1750] The device packages the basic information entered in JSON format and sends it to the server.

[1751] Step 2: Register your face photo and enter your beauty style

[1752] Behavior 2.1

[1753] Users can upload a photo of their face to the app using their device's camera or by selecting a suitable photo from their gallery.

[1754] Behavior 2.2

[1755] The user selects the desired makeup and hairstyle (e.g., natural makeup, bob cut).

[1756] Behavior 2.3

[1757] The device compiles the facial photo and beauty style information into JSON format and sends it to the server.

[1758] Step 3: Emotion recognition by the emotion engine

[1759] Behavior 3.1

[1760] The server analyzes the received facial photo using an emotion engine to recognize the user's emotions.

[1761] Behavior 3.2

[1762] The server stores the recognized emotion information in a database and prepares it for further processing.

[1763] Step 4: Comprehensive production by AI

[1764] Behavior 4.1

[1765] The server launches an artificial intelligence algorithm to analyze basic information, emotional information, facial photos, and desired beauty style information.

[1766] Behavior 4.2

[1767] The server's artificial intelligence algorithm generates optimal fashion, makeup, and hairstyle suggestions based on the original input information and emotional information.

[1768] Behavior 4.3

[1769] The server packages the generated proposal results in JSON format and sends them to the terminal.

[1770] Step 5: Review your proposal and provide further information

[1771] Operation 5.1

[1772] The device analyzes the suggestions received from the server and displays visual suggestions on the app interface.

[1773] Behavior 5.2

[1774] The user reviews the suggestions and, if they like them, clicks the button to display more information.

[1775] Operation 5.3

[1776] The device displays a list of suggested fashion and beauty items, as well as information on where to buy them and beauty salons.

[1777] Step 6: Purchase and reservation assistance

[1778] Operation 6.1

[1779] If the user wishes to purchase the suggested item or make a salon appointment, they click on the displayed link.

[1780] Operation 6.2

[1781] Based on the link clicked, the device redirects the user to the purchase website or beauty salon reservation system.

[1782] Operation 6.3

[1783] The server records the user's purchasing and reservation behavior and stores it in an economic behavior database.

[1784] Step 7: Providing advice on skin type and hair concerns

[1785] Operation 7.1

[1786] Users can also enter their skin type and hair concerns (e.g., dry skin, dandruff, etc.) into the app.

[1787] Operation 7.2

[1788] The device packages the entered information about skin type and hair concerns in JSON format and sends it to the server.

[1789] Operation 7.3

[1790] The server uses artificial intelligence algorithms to generate appropriate advice and a list of related products based on the input skin type and hair concerns, while also taking emotional information into account.

[1791] Operation 7.4

[1792] The server packages the generated advice and product list in JSON format and sends it to the terminal.

[1793] Operation 7.5

[1794] The device displays the received advice and a list of related products on the app interface.

[1795] Specific examples

[1796] Example 1: Entering basic information and beauty information

[1797] A 35-year-old female user logs into the app and enters her age, gender, height 160cm, weight 55kg, skin color light beige, and budget of 20,000 yen. Next, she uploads a photo of her face and selects her desired style of natural makeup and bob cut. The device compiles this information into JSON format and sends it to the server.

[1798] Example 2: Emotion Recognition with an Emotion Engine

[1799] The server analyzes the received facial photo, and the emotion engine recognizes the user's emotion as "happy." This emotion information is used to tailor the suggestions.

[1800] Example 3: Proposal generation and display

[1801] Based on the input basic information, desired style information, and emotional information, the server generates style suggestions, such as autumn casual fashion, natural makeup, and bob cuts, suitable for women of the same age. The results of these suggestions are sent to the device as visuals. The device displays the suggestions to the user, who can click a button to view more information.

[1802] Example 4: Purchasing and reservation assistance

[1803] The user likes the suggested items, a blouse and a skirt, and clicks on a link to purchase them. The device redirects the user to the website where the items are purchased, and the server records this action.

[1804] By following these steps, the system based on the present invention provides users with customized fashion and beauty suggestions that take emotional information into account, creating an environment for quick purchases and reservations.

[1805] Example 2

[1806] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1807] Current beauty and fashion recommendation systems often do not adequately consider users' emotions or detailed personal information. As a result, the recommendations provided may not meet users' expectations. Furthermore, many systems do not include a means to smoothly complete purchase or reservation procedures, resulting in a lack of convenience for users. This poses a challenge, resulting in a decrease in user satisfaction.

[1808] The identification process by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes: means for inputting basic information about the user; means for inputting a facial photo and desired beauty style information about the user; artificial intelligence algorithm means for analyzing the input information and making optimal fashion, makeup, and hairstyle suggestions; emotion recognition means for analyzing the received facial photo and recognizing the user's emotions; means for generating suggestions and adjusting them based on the emotion information; means for providing information about where to purchase the suggested fashion and beauty items and beauty salons; and means for providing appropriate advice and a list of related products based on the user's skin type or hair concerns. This makes it possible to provide more personalized suggestions that take into account the user's emotions and detailed personal information, facilitating smoother purchasing and reservation procedures.

[1809] "Basic information" refers to personal information such as the user's age, gender, height, weight, skin color, and budget.

[1810] "Face Photo" refers to an image of a user's face uploaded by the user.

[1811] "Beauty style information" refers to information about the user's desired beauty style, such as desired makeup and hairstyle.

[1812] "Artificial intelligence algorithm" refers to an algorithm that analyzes input information and generates optimal fashion suggestions, makeup suggestions, and hairstyle suggestions.

[1813] "Emotion recognition means" refers to a system that analyzes the received facial photo and recognizes the user's emotions.

[1814] The "means for adjusting the recommendation result" refers to a means for adjusting the generated recommendation result based on the recognized emotion.

[1815] "Purchaser Information" refers to information about where to purchase suggested fashion and beauty items.

[1816] "Beauty salon information" refers to information about a beauty salon that provides the proposed beauty services.

[1817] "Skin type" refers to the nature and condition of the user's skin.

[1818] "Hair concerns" refers to the hair-related problems and hopes that users have.

[1819] "Advice means" refers to a means for providing appropriate advice based on the user's skin type and hair concerns.

[1820] "Related Product Listings" refers to lists of beauty products or items recommended based on the advice provided.

[1821] "Means for inputting basic information" refers to the interface and functions that allow users to input basic information.

[1822] "Means for inputting face photo and beauty style information" refers to an interface or function that allows a user to input a face photo and desired beauty style information.

[1823] The present invention relates to a system that integrates a user's basic information, facial photo, desired beauty style information, and emotional information to provide optimal fashion, makeup, and hairstyle suggestions. This system can personalize suggestions using an emotion engine that recognizes the user's emotions and an artificial intelligence algorithm. The specific configuration and operation of the present invention are described below.

[1824] composition

[1825] 1. Enter and submit user information

[1826] The user launches the application and enters basic information such as age, gender, height, weight, skin color, and budget. The device packages the entered basic information in JSON format and sends it to the server. The software used is an interface for entering basic information and a program for converting it to JSON format.

[1827] 2. Register your face photo and enter your beauty style

[1828] The user then uploads a photo of their face and selects the desired makeup and hairstyle. This is also compiled into JSON format by the device and sent to the server. The software used is an interface for uploading face photos and selecting beauty styles, as well as a comprehensive data package program.

[1829] 3. Emotion Recognition by Emotion Engine

[1830] The server analyzes the received facial photos and recognizes the user's emotions using an emotion engine. This emotion information is used to adjust fashion and beauty suggestions. The hardware used is a high-performance image analysis server, and the software is an emotion recognition engine (e.g., Google Cloud Vision API).

[1831] 4. Comprehensive production by AI

[1832] The server analyzes the received basic information, emotional information, facial photo, and desired beauty style information, and uses artificial intelligence algorithms to comprehensively produce the most suitable fashion, makeup, and hairstyle for the user. This information is generated as an integrated visual proposal, which is adjusted based on the emotional information. The proposal results are sent from the server to the device. The software used is a machine learning model (e.g., TensorFlow) and an image generation algorithm.

[1833] 5. Confirmation of proposal and provision of further information

[1834] The device analyzes the recommendation results received from the server and displays visual suggestions on the app's interface. The user checks the suggestions and clicks the display button if more information is needed. This displays information on where to purchase the suggested fashion and beauty items and beauty salons. The software used is the recommendation result display interface and the detailed information display program.

[1835] 6. Purchase and reservation assistance

[1836] If the user wishes to purchase the suggested item or make a salon reservation, they operate the device and click on the link. This causes the device to redirect the user to the purchasing website or salon reservation system. The server records this behavior and stores it in an economic behavior database. The software used is a redirect program and a behavior recording program.

[1837] 7. Providing advice on skin type and hair problems

[1838] Additionally, users can enter their skin type and hair concerns. The device sends this information to a server, which uses AI to generate appropriate advice and a list of related products. Emotional information is taken into account to provide tailored advice. The device then displays the received advice and product list to the user. The software used is a skin type and hair concern input interface and a program that generates a list of related products.

[1839] Specific examples

[1840] Example 1: Entering basic information and beauty information

[1841] A 35-year-old female user logs into the app and enters her age, gender, height 160cm, weight 55kg, skin color light beige, and budget of 20,000 yen. Next, she uploads a photo of her face and selects her desired style of natural makeup and bob cut. The device compiles this information into JSON format and sends it to the server.

[1842] Example 2: Emotion Recognition with an Emotion Engine

[1843] The server analyzes the received facial photo, and the emotion engine recognizes the user's emotion as "happy." This emotion information is used to tailor the suggestions.

[1844] Example 3: Proposal generation and display

[1845] Based on the user's basic information, desired style, and emotional information, the server proposes styles suitable for women of the same age, such as autumn casual fashion, natural makeup, and bob cuts. The proposed results are sent to the device as visuals. The device displays the proposed content to the user, who can click a button to view more information.

[1846] Example 4: Purchasing and reservation assistance

[1847] The user likes the suggested items, a blouse and a skirt, and clicks on a link to purchase them. The device redirects the user to the website where the items are purchased, and the server records this action.

[1848] Prompt Sentence Examples

[1849] "Please suggest something that suits the casual fashion and natural makeup worn by a 35-year-old woman. Furthermore, the user is showing the emotion 'happy.'"

[1850] This allows users to get the best fashion and beauty suggestions from the generative AI model.

[1851] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1852] Step 1:

[1853] A user launches the application and logs in. They then enter basic information such as age, gender, height, weight, skin tone, and budget.

[1854] Input: Basic information entered by the user.

[1855] Specific operation: User authentication is performed on the application login screen, and after moving to the main screen, a basic information input form is displayed. When the user clicks the submit button, the input data is converted to JSON format on the terminal side.

[1856] Output: Basic information converted to JSON format.

[1857] Step 2:

[1858] The terminal transmits the converted basic information to the server.

[1859] Input: Basic information in JSON format.

[1860] Specific operation: The device sends JSON data to the server using the REST API.

[1861] Output: Basic information data received by the server.

[1862] Step 3:

[1863] Users upload a photo of their face and select the desired makeup and hairstyle.

[1864] Input: A photo of the user's face and desired beauty style.

[1865] Specific operation: A face photo upload screen is displayed, the user selects a photo, clicks the upload button, and after selecting a beauty style, the device compiles this information into JSON format.

[1866] Output: A JSON-formatted face photo and desired beauty style information.

[1867] Step 4:

[1868] The device sends a facial photo and beauty style information in JSON format to the server.

[1869] Input: Face photo and beauty style information in JSON format.

[1870] Specific operation: The device sends data to the server using the REST API.

[1871] Output: Face photo and beauty style information data received by the server.

[1872] Step 5:

[1873] The server analyzes the received facial photograph and recognizes the user's emotions using an emotion recognition engine.

[1874] Input: Facial photo data.

[1875] Specific operation: The server uses a high-performance image analysis server to run a facial recognition algorithm and analyzes emotions using an emotion recognition engine (e.g., Google Cloud Vision API).

[1876] Output: Emotion information data.

[1877] Step 6:

[1878] The server analyzes basic information, emotional information, facial photos, and desired beauty style information, and uses artificial intelligence algorithms to produce the optimal fashion, makeup, and hairstyle.

[1879] Input: Basic information, emotional information, face photo, desired beauty style information.

[1880] How it works: The server analyzes this information using a data analysis module and generates suggestions using a machine learning model (e.g., TensorFlow), which are then visualized using an image generation algorithm.

[1881] Output: Visual suggestion data.

[1882] Step 7:

[1883] The server transmits the visual suggestion data to the terminal.

[1884] Input: Visual suggestion data.

[1885] Specific operation: The server sends visual suggestion data to the device using the REST API.

[1886] Output: Visual suggestion data received by the device.

[1887] Step 8:

[1888] The device analyzes the received suggestions and displays visual suggestions on the app interface. The user can review the suggestions and click the view button to view more information.

[1889] Input: Visual suggestion data.

[1890] What happens: The device parses the received content and displays it in the app's UI as images and text. When the user clicks the details button, the additional information expands to show more information.

[1891] Output: Visual suggestions and detailed information displayed in the app's UI.

[1892] Step 9:

[1893] If the user wishes to purchase the suggested item or make a salon reservation, he or she operates the terminal and clicks on the link.

[1894] Input: Purchase or reservation link.

[1895] What happens: When a user clicks on a purchase link, their device's browser opens to the website where the purchase is made.

[1896] Output: Purchasing website or salon booking system.

[1897] Step 10:

[1898] The server records user behavior and stores it in an economic behavior database.

[1899] Input: User behavior data.

[1900] Specific operation: The server logs users' link clicks and purchase behavior and stores them in an economic behavior database.

[1901] Output: Stored behavioral data.

[1902] Step 11:

[1903] The user enters additional information such as their skin type and hair concerns, and the device sends this to the server.

[1904] Input: Skin type and hair concerns.

[1905] Specific behavior: When the user fills out the additional information input form and clicks the submit button, the device converts the data into JSON format and sends it to the server.

[1906] Output: Skin type and hair problem information received by the server.

[1907] Step 12:

[1908] The server uses AI to generate appropriate advice and a list of related products based on skin type and hair concerns.

[1909] Input: Skin type and hair concerns.

[1910] How it works: The server uses an AI model to analyze the input information and generate advice and product lists.

[1911] Output: Advice and related product list.

[1912] Step 13:

[1913] The terminal displays the received advice and related product list to the user.

[1914] Input: Advice and related product listings.

[1915] Specific behavior: The device parses the received content and displays it in the app's UI.

[1916] Output: Advice and related product list displayed in the app UI.

[1917] (Application example 2)

[1918] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1919] Today's consumers demand personalized fashion and beauty recommendations, but existing systems often only offer one-size-fits-all recommendations, resulting in low satisfaction. Furthermore, the effectiveness of recommendations is limited because they are not tailored based on real-time emotional information. Furthermore, there is a lack of support for in-store purchases and try-ons, creating a need for ways to improve the user shopping experience.

[1920] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes: means for inputting basic information about the user; means for inputting a facial photo and desired beauty style information about the user; means including an artificial intelligence algorithm for analyzing the input information and making optimal fashion, makeup, and hairstyle suggestions; means for providing information on where to purchase the suggested fashion and beauty items and beauty salon information; means for providing appropriate advice and a list of related products based on the user's skin type or hair concerns; means for displaying the suggestion results on smart glasses worn by the user in a physical store, recognizing emotions in real time, and adjusting the suggestion content based on the emotional information; and means for supporting the user in trying on and purchasing items they like through the smart glasses. This enables the provision of personalized suggestions and adjustments based on real-time emotional information, as well as an improved shopping experience in a physical store.

[1921] "Means for inputting basic user information" refers to devices or applications for inputting basic information such as the user's age, gender, height, weight, skin color, and budget.

[1922] The "means for inputting the user's facial photograph and desired beauty style information" refers to a device or application for inputting the user's facial photograph and beauty style information such as desired makeup and hairstyle.

[1923] "Means including an artificial intelligence algorithm for analyzing input information and...

Claims

1. A means for inputting basic information of a user; A means for inputting a user's face photo and desired beauty style information; means including an artificial intelligence algorithm for analyzing input information and making optimal fashion suggestions, makeup suggestions, and hairstyle suggestions; A means for providing information on where to purchase the suggested fashion and beauty items and information on beauty salons; A means for providing appropriate advice and a list of related products based on the user's skin type or hair concerns; A system including:

2. The system of claim 1 , further comprising means for redirecting a prospective purchaser to a suggested retailer or beauty salon.

3. The system of claim 1 , further comprising means for recording and storing economic behavior data of users.

Images