System

Abstract

A system is provided.SOLUTION: A system comprising: means for receiving profile information from a user; means for receiving a facial image of the user and analyzing a skeleton and a personal color through image analysis; means for interactively posing questions to the user and analyzing answers from the user; means for integrating the received profile information, the facial image analysis results, and the user answers to generate comprehensive diagnostic results using a AI model; and means for suggesting fashion items, brands, and shops suitable for the user based on the diagnostic results.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] Traditionally, there have been several issues with personal color diagnosis and styling advice to find fashion and makeup items that suit you. Hiring a professional can be expensive, while simple online diagnosis has low accuracy. There is a need for a method that solves these issues and allows many people to easily and accurately find items that suit them. [Means for solving the problem]

[0005] This invention is a system including: means for receiving profile information from a user; means for receiving a facial image of the user and analyzing the bone structure and personal color through image analysis; means for interactively asking the user questions and analyzing the user's responses; means for integrating the received profile information, facial image analysis results, and user responses and generating a comprehensive diagnostic result using an AI model; and means for suggesting fashion items, brands, and shops suitable for the user based on the diagnostic result. This system allows users to easily find fashion items and makeup items that suit them with high accuracy.

[0006] A "user" is an individual who uses this system with the goal of finding fashion and makeup items that suit them.

[0007] "Profile Information" means basic personal information provided by a User, including information such as age, gender, preferred fashion style, etc.

[0008] A "face image" is a photograph of the user's face uploaded by the user, and is image data used for analyzing bone structure and personal color.

[0009] "Image analysis" is the process of using algorithms on facial images to identify bone structure and personal colors.

[0010] "Personal colors" are a group of colors that best suit a user based on their skin color, eye color, hair color, etc.

[0011] "Dialogue questions" are questions that the system asks the user in succession, and are a means of gaining a detailed understanding of the user's preferences and characteristics based on the answers.

[0012] An "AI model" is a model that includes a trained machine learning algorithm and generates a comprehensive diagnostic result based on profile information, facial image analysis results, and user responses.

[0013] "Comprehensive diagnosis results" are diagnostic results generated by an AI model that identify the fashion items, brands, shops, etc. that are best suited to the user.

[0014] "Fashion items" are products such as clothes, accessories, shoes, and bags worn by users.

[0015] A "brand" is a group of products offered by a particular manufacturer or designer, and is part of a range of fashion items.

[0016] "Shop" means a physical store or online store where users can purchase suggested fashion items. [Brief explanation of the drawings]

[0017] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8]FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 3 is a sequence diagram showing a processing flow of the data processing system according to the first embodiment. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0018] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0019] First, the terms used in the following description will be explained.

[0020] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0021] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0022] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0023] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0024] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0025] [First embodiment]

[0026] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0027] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0028] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0029] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0030] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0031] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0032] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0033] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0034] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0035] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0036] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0037] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0038] MODE FOR CARRYING OUT THE INVENTION

[0039] System program and processing description

[0040] The system of this invention utilizes AI to help users find fashion and makeup items that suit them. The system operates in cooperation with a server, a terminal, and a user, each of which plays a specific role.

[0041] Server Operation

[0042] 1. Receiving profile information:

[0043] The server receives the user's profile information (age, gender, fashion preferences, etc.) sent from the device and stores it in a database.

[0044] 2. Receiving and analyzing face images:

[0045] The server receives the user's facial image from the device, applies an image analysis algorithm to analyze the user's bone structure and personal color, and temporarily stores the facial image analysis results.

[0046] 3. Interactive question generation and answer analysis:

[0047] The server generates interactive questions and asks them to the user via the terminal, receives the user's answers, analyzes the answers, and dynamically generates the next questions to ask. This process is repeated until all necessary information is obtained.

[0048] 4. Generating comprehensive diagnostic results:

[0049] The server integrates the received profile information, facial image analysis results, and user responses, and uses an AI model to generate a comprehensive diagnosis, which includes suitable fashion items, brands, and shops for the user.

[0050] 5. Presentation of results and recommendations:

[0051] Based on the generated diagnostic results, the server generates suggestions for specific fashion items, brands, and shops, and transmits them to the terminal to present to the user.

[0052] Device behavior

[0053] 1. Provide profile information input interface:

[0054] The terminal provides an interface for the user to input profile information and transmits the input information to the server.

[0055] 2. Provide face image upload interface:

[0056] The terminal provides an interface for the user to upload a facial image and transmits the uploaded facial image to the server.

[0057] 3. Providing an interactive question and answer input interface:

[0058] The terminal displays the questions sent from the server, provides an interface for the user to answer the questions, and sends the user's answers to the server.

[0059] 4. Provide diagnostic result display interface:

[0060] The terminal provides an interface for displaying the diagnosis results and suggestions sent from the server to the user.

[0061] User behavior

[0062] 1. Enter your profile information:

[0063] Through the device interface, users enter their profile information, including age, gender, and fashion preferences.

[0064] 2. Upload your face image:

[0065] Users upload their facial images using the device's interface.

[0066] 3. Answer the interactive questions:

[0067] The user answers questions sent from the server through the device. For example, to the question "What color clothes do you like?", the user answers "I like pastel colors."

[0068] 4. Review the diagnostic results and accept the recommendations:

[0069] Users can check the diagnostic results displayed on their device and browse suggested fashion items, brands, and shops.

[0070] Specific examples

[0071] Example 1: A woman in her 30s who likes casual fashion

[0072] 1. User: Accesses the app and enters age (30s), gender (female), and fashion preference (casual).

[0073] 2. Terminal: Sends input information to the server.

[0074] 3. User: Upload a photo of your face.

[0075] 4. Device: Send a photo of your face to the server.

[0076] 5. Server: Performs facial image analysis and determines the user is a "spring type." Generates a dialogue question, asking, "What color clothes do you like?"

[0077] 6. User: "I like pastel colors."

[0078] 7. Terminal: Sends the answer to the server.

[0079] 8. Server: Based on all the information, identify pastel-colored casual fashion items and suitable brands and generate a diagnosis.

[0080] 9. Terminal: Display the diagnostic results.

[0081] 10. User: Select an item based on the diagnosis results and purchase it from the suggested shop.

[0082] Example 2: A man in his 40s who likes formal fashion

[0083] 1. User: Accesses the app and enters age (40s), gender (male), and fashion preference (formal).

[0084] 2. Terminal: Sends input information to the server.

[0085] 3. User: Upload a photo of your face.

[0086] 4. Device: Send a photo of your face to the server.

[0087] 5. Server: Performs facial image analysis and determines the person is a "winter type." Generates a dialogue question, asking, "What color do you prefer for formal occasions?"

[0088] 6. User: "Dark blue or black."

[0089] 7. Terminal: Sends the answer to the server.

[0090] 8. Server: Based on all the information, it identifies formal fashion items and popular brands and generates diagnostic results.

[0091] 9. Terminal: Display the diagnostic results.

[0092] 10. User: Select an item based on the diagnosis results and purchase it from the suggested shop.

[0093] In this way, users can easily find the fashion style that suits them through highly accurate diagnosis. The system handles everything from entering profile information, analyzing facial images, asking interactive questions, and generating and proposing comprehensive diagnosis results.

[0094] The processing flow will be explained below.

[0095] Program processing steps

[0096] Step 1: Enter your user information

[0097] User: Visits an app or website and enters profile information such as age, gender, and fashion preferences.

[0098] Terminal: Receives profile information entered by the user and sends it to the server.

[0099] Server: Stores the received profile information in a database and prepares for the next step.

[0100] Step 2: Upload and analyze face images

[0101] Users: Upload a photo of themselves to an app or website.

[0102] Terminal: Sends the uploaded face photo to the server.

[0103] Server: Analyzes bone structure and personal color using facial image analysis algorithms. Saves the analysis results temporarily and uses them in the next step.

[0104] Step 3: Interactive question generation and answer analysis

[0105] Server: Generates interactive questions and sends them to the device. Initial questions include "What color clothes do you like?" and "What kind of clothes do you like to wear for what occasions?"

[0106] Terminal: Provides an interface for displaying interactive questions to the user and for entering answers.

[0107] User: Enters an answer to a question.

[0108] Terminal: Sends the entered answer to the server.

[0109] Server: Analyzes the received answers and dynamically generates the next question to ask. This process is repeated until all necessary questions have been asked.

[0110] Step 4: Generate comprehensive diagnostic results

[0111] Server: The server combines the results of facial image analysis with the user's responses to interactive questions and uses an AI model to generate a comprehensive diagnosis, specifically identifying the user's personal color, clothing styles and designs that suit them, popular brands, and suitable accessories and makeup items.

[0112] Server: Stores the diagnostic results in a database and prepares the results for display.

[0113] Step 5: Present the results and make recommendations

[0114] Server: Generates the diagnosis results and specific fashion item, brand, and shop recommendations based on them. The recommendations may be updated in real time, so they must be generated dynamically.

[0115] Terminal: Provides an interface for presenting diagnostic results to the user.

[0116] User: Based on the results presented, users can choose fashion and makeup items that suit them best.

[0117] Specific examples

[0118] Example: A woman in her 30s who likes casual fashion

[0119] 1. Step 1:

[0120] User: Accesses the app and enters age (30s), gender (female), and fashion preference (casual).

[0121] Terminal: Sends input information to the server.

[0122] Server: Saves the input information in a database.

[0123] 2. Step 2:

[0124] User: Upload a photo of themselves to the app.

[0125] Device: Sends a photo of your face to the server.

[0126] Server: Performs facial image analysis and determines "spring type." Temporarily saves the analysis results.

[0127] 3. Step 3:

[0128] Server: Generates a dialogue question and asks the user, "What color clothes do you like?"

[0129] Terminal: Displays questions and accepts user answers.

[0130] User: "I like pastel colors."

[0131] Terminal: Sends the answer to the server.

[0132] Server: Analyzes the answer and generates the next question: "What kind of clothes would you like to wear for what occasion?"

[0133] Terminal: Show question.

[0134] User: Enter "Relaxing holiday scene."

[0135] Device: Sends the answer to the server, analyzes the answer, and repeats this process until it has gathered all the information it needs.

[0136] 4. Step 4:

[0137] Server: Using an AI model based on the results of facial image analysis and answers to interactive questions, the server generates diagnostic results suitable for casual fashion, including information on "pastel-colored items" and "suitable brands."

[0138] Server: Stores the diagnostic results in a database.

[0139] 5. Step 5:

[0140] Server: Dynamically generates diagnostic results and recommendations for specific fashion items, brands, and shops based on those results.

[0141] On the device: Display diagnostic results and suggestions to the user.

[0142] User: Based on the presented results, select a pastel-colored casual fashion item and purchase it from the suggested shop.

[0143] In this way, users can receive highly accurate and realistic fashion advice throughout the process.

[0144] Example 1

[0145] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0146] Users often spend a lot of time and effort finding the perfect fashion and makeup items for themselves, and it can be particularly difficult to choose items that take into account facial features and personal color. Current systems are unable to accurately analyze these points and make real-time suggestions that match the user's preferences and characteristics. This leaves users with the challenge of being unable to quickly and easily find the perfect fashion and makeup items based on their preferences and characteristics.

[0147] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0148] In this invention, the server includes means for receiving profile information from a user, means for receiving a facial image of the user and analyzing bone structure and personal color through image analysis, means for interactively asking the user questions and analyzing the user's responses, means for integrating the received profile information, facial image analysis results, and user responses and generating a comprehensive diagnostic result using an artificial intelligence model, means for suggesting clothing items, brands, and retailers suitable for the user based on the diagnostic result, communication means for sending initial information to the server in JSON format, means for sending a facial image via an HTTP request, means for analyzing the facial image using an image analysis library, means for dynamically generating interactive questions using the profile information and image analysis results, and means for generating and presenting a comprehensive diagnostic result, thereby enabling users to quickly and easily find fashion and makeup items based on their characteristics and preferences.

[0149] "Profile information" refers to information about a user's personal characteristics, such as the user's age, gender, and fashion preferences.

[0150] A "face image" is an image of the user's face, and facial features and personal color are analyzed based on this image.

[0151] "Image analysis" is a process performed on a received facial image to identify the user's bone structure and personal color.

[0152] "Dialogue format" refers to a format in which questions are asked to the user sequentially, and the next question is dynamically generated based on the answers.

[0153] An "artificial intelligence model" is a model that uses technologies such as machine learning and deep learning, and generates comprehensive diagnostic results based on the received data.

[0154] The "comprehensive diagnosis results" are generated by integrating profile information, facial image analysis results, and user responses, and include suggestions for fashion items, trademarks, and retailers that are suitable for the user.

[0155] "Communication means" refers to the technical means for transmitting initial information and analysis data to the server in JSON format or via HTTP requests.

[0156] The "JSON format" is a format that structures and expresses data in text format, and is primarily used for sending and receiving data.

[0157] An "HTTP request" is a protocol that allows a client to request a server to send data or obtain information.

[0158] An "image analysis library" is a software library, such as OpenCV or Dlib, that is used to extract and analyze features from facial images.

[0159] "Dynamic generation" means adaptively generating the next question or suggestion on the spot based on the user's answers.

[0160] "Diagnosis results" are recommendations for fashion items and services suitable for the user, generated based on analysis of the received data and artificial intelligence models.

[0161] MODE FOR CARRYING OUT THE INVENTION

[0162] This invention provides a system in which a server, a terminal, and a user work together to help users find the fashion and makeup items that are best suited to them. The overall configuration of the system and the operation of each component are described in detail below.

[0163] Server Operation

[0164] 1. Receiving and storing profile information:

[0165] The server receives the user's profile information (age, gender, fashion preferences, etc.) sent from the device and stores it in a database (e.g., MySQL or PostgreSQL), making it possible to manage each user's individual information.

[0166] 2. Receiving and analyzing face images:

[0167] After receiving the user's facial image from the device, the server analyzes the user's bone structure and personal color using image analysis libraries such as OpenCV and Dlib. The analysis results are temporarily stored in a database.

[0168] 3. Interactive question generation and answer analysis:

[0169] The server uses an NLP library (e.g., NLTK or SpaCy) to generate interactive questions and pose them to the user through the terminal. After receiving the user's answers, it analyzes their content and dynamically generates the next questions.

[0170] 4. Generating comprehensive diagnostic results:

[0171] The server integrates the profile information, facial image analysis results, and user responses, and generates a comprehensive diagnostic result using an AI model (e.g., TensorFlow or PyTorch), which can identify suitable fashion and makeup items for the user.

[0172] 5. Presentation of results and recommendations:

[0173] The server generates specific suggestions based on the generated diagnostic results and sends them to the terminal for presentation to the user, including clothing items, brands, and retailers suitable for the user.

[0174] Device behavior

[0175] 1. Provide profile information input interface:

[0176] The terminal provides an interface for the user to input profile information and transmits the input information to the server.

[0177] 2. Provide face image upload interface:

[0178] The terminal provides an interface for the user to upload a facial image and transmits the uploaded facial image to the server.

[0179] 3. Providing an interactive question and answer input interface:

[0180] The terminal displays the questions sent from the server, provides an interface for the user to answer the questions, and sends the user's answers to the server.

[0181] 4. Provide diagnostic result display interface:

[0182] The terminal provides an interface for displaying the diagnosis results and suggestions sent from the server to the user.

[0183] User behavior

[0184] 1. Enter your profile information:

[0185] Users enter their profile information through the device interface, including their age, gender and fashion preferences.

[0186] 2. Upload your face image:

[0187] Users upload their facial images using the device's interface.

[0188] 3. Answer the interactive questions:

[0189] The user answers questions sent from the server through the device. For example, to the question "What color clothes do you like?", the user answers "I like pastel colors."

[0190] 4. Review the diagnostic results and accept the recommendations:

[0191] Users can check the diagnosis results displayed on their device and use the suggested fashion and makeup items as a reference.

[0192] Specific use cases

[0193] Example 1: A woman in her 30s who likes casual fashion

[0194] 1. User: Accesses the app and enters age (30s), gender (female), and fashion preference (casual).

[0195] 2. Terminal: Sends input information to the server.

[0196] 3. User: Upload a photo of yourself.

[0197] 4. Device: Sends a photo of your face to the server.

[0198] 5. Server: Performs facial image analysis and determines the user is a "spring type." Generates a dialogue question, asking, "What color clothes do you like?"

[0199] 6. User: "I like pastel colors."

[0200] 7. Terminal: Sends the answer to the server.

[0201] 8. Server: Based on all the information, it identifies pastel-colored casual fashion items and suitable trademarks and generates a diagnosis result.

[0202] 9. Terminal: Display the diagnostic results.

[0203] 10. User: Select an item based on the diagnosis results and purchase it from the suggested retailer.

[0204] Example 2: A man in his 40s who likes formal fashion

[0205] 1. User: Accesses the app and enters age (40s), gender (male), and fashion preference (formal).

[0206] 2. Terminal: Sends input information to the server.

[0207] 3. User: Upload a photo of yourself.

[0208] 4. Device: Sends a photo of your face to the server.

[0209] 5. Server: Performs facial image analysis and determines the person is a "winter type." Generates a dialogue question, asking, "What color do you prefer for formal occasions?"

[0210] 6. User: "I like dark blue and black."

[0211] 7. Terminal: Sends the answer to the server.

[0212] 8. Server: Based on all the information, it identifies formal fashion items and popular brands and generates diagnostic results.

[0213] 9. Terminal: Display the diagnostic results.

[0214] 10. User: Select an item based on the diagnosis results and purchase it from the suggested retailer.

[0215] In this way, users can quickly and easily find the fashion and makeup items that best suit them through the system.

[0216] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0217] Step 1: Enter your profile information

[0218] User Action: The user enters their profile information (age, gender, fashion preferences) into the interface provided by the device.

[0219] Input: User profile information (e.g., "30s," "Female," "Casual")

[0220] Output: Sending profile information from device to server

[0221] Specific action: Enter data into a form in a web application and press the "Submit" button.

[0222] Step 2: Submit your profile information

[0223] Device behavior: The device sends the entered profile information to the server in JSON format.

[0224] Input: Profile information entered by the user

[0225] Output: Profile information received by the server

[0226] What it does: Sends profile information to the / api / profile endpoint using an HTTP POST request.

[0227] Step 3: Upload your face image

[0228] User action: The user selects and uploads a face image using the device interface.

[0229] Input: A face image selected by the user

[0230] Output: Sending face image from device to server

[0231] Specific operation: Select an image file and press the "Upload" button.

[0232] Step 4: Send a face image

[0233] Device operation: The device sends the uploaded facial image to the server.

[0234] Input: Face image file uploaded by the user

[0235] Output: Face image received by the server

[0236] What it does: Sends an image file to the / api / upload endpoint using an HTTP POST request.

[0237] Step 5: Facial image analysis

[0238] Server operation: The server analyzes the received facial image using an image analysis library such as OpenCV or Dlib to identify bone structure and personal color.

[0239] Input: Face image sent to the server

[0240] Output: Analysis results of bone structure and personal color

[0241] Specific operation: Run the Python script and use the analyze_face(image) function to extract and identify features from the facial image.

[0242] Step 6: Interactive question generation and answer collection

[0243] Server operation: The server uses an NLP library to generate interactive questions based on the collected information and sends the questions to the user via the terminal.

[0244] Terminal operation: The terminal displays questions sent from the server and sends answers from the user to the server.

[0245] User action: The user answers questions sent by the server through the terminal.

[0246] Input: Server-generated question, user-entered answer

[0247] Output: The user's answer received by the server

[0248] Specific action: In response to the question "What color clothes do you like?", enter "I like pastel colors" as the answer and submit.

[0249] Step 7: Generate comprehensive diagnostic results

[0250] Server operation: The server integrates the user's profile information, facial image analysis results, and answers to interactive questions, and uses an AI model to generate a comprehensive diagnosis result.

[0251] Input: User profile information, facial image analysis results, answers to dialogue questions

[0252] Output: Comprehensive diagnostic results (suitable fashion items, brands, and retailers)

[0253] Specific operation: Using the TensorFlow model, generate diagnostic results with the generate_recommendations(profile, face_analysis, responses) function.

[0254] Step 8: Send and view results

[0255] Server operation: The server sends the generated diagnostic results to the terminal.

[0256] Device behavior: The device displays the received diagnostic results to the user.

[0257] User Action: User reviews diagnostic results and views suggested items.

[0258] Input: Server-generated diagnostic results

[0259] Output: Diagnostics displayed on the terminal

[0260] How it works: The diagnosis results are displayed on the device screen, and the user can check out the suggested fashion and makeup items.

[0261] In this way, users can quickly and easily find the fashion and makeup items that best suit them through the system.

[0262] (Application example 1)

[0263] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0264] With so many options available, it can be difficult for users to find the right fashion or makeup items. There is also a need for the ability to try on items without actually going to a store. However, the current lack of appropriate technology to make this a reality is problematic. Furthermore, there is a growing need for systems that can dynamically reflect users' preferences and characteristics in real time and make highly accurate recommendations.

[0265] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0266] In this invention, the server includes means for receiving profile information from a user, means for receiving a facial image of the user and analyzing the image to analyze bone structure and personal color, means for interactively asking the user questions and analyzing the user's responses, means for integrating the received profile information, facial image analysis results, and user responses and generating a comprehensive diagnostic result using an AI model, means for suggesting fashion items, brands, and stores suitable for the user based on the generated diagnostic result, and means for capturing images of the user's face and body using smart glasses to provide a virtual try-on experience, allowing the user to try on fashion items in a realistic way without actually going to a store and receiving highly accurate suggestions for fashion items.

[0267] "Profile information" refers to information such as a user's age, gender, and fashion preferences.

[0268] "Facial Image" refers to a photograph or video of a user's face.

[0269] "Image analysis" refers to the process of analyzing received facial images to extract features such as bone structure and personal color.

[0270] An "AI model" refers to an algorithm that uses machine learning or deep learning to analyze data and generate a specific diagnostic result.

[0271] "Comprehensive diagnosis results" refer to results that indicate the fashion items and brands that are best suited to a user, generated by integrating profile information, facial image analysis results, and the user's responses.

[0272] "Dialogue-style questions" refers to a format in which the system asks the user a series of questions and collects the user's answers.

[0273] "Smart glasses" refers to a glasses-type device that, when worn by a user, provides functions such as augmented reality and displays information in the user's field of vision.

[0274] A "virtual try-on experience" is an experience that allows users to get the feeling of trying on clothes in a virtual space without actually trying them on.

[0275] The system embodying this invention utilizes AI to help users find fashion and makeup items that suit them. The specific configuration of the system and its operation method are described below.

[0276] Server Operation

[0277] 1. Receiving profile information

[0278] The server receives the user's profile information (age, gender, fashion preferences, etc.) sent from the device and stores it in a database.

[0279] 2. Receiving and analyzing face images

[0280] After receiving the user's facial image from the device, the server applies image analysis algorithms to analyze the user's bone structure and personal color. The facial image analysis uses OpenCV and Keras models. The analysis results are temporarily stored.

[0281] 3. Interactive Question Generation and Answer Analysis

[0282] The server generates interactive questions and asks them to the user via the device, then uses an AI model to analyze the user's answers and dynamically generate the next questions until the required information is obtained.

[0283] 4. Generating comprehensive diagnostic results

[0284] The server integrates the received profile information, facial image analysis results, and the user's responses, and uses an AI model to generate a comprehensive diagnosis, which includes suitable fashion items, brands, and stores for the user.

[0285] 5. Presentation of results and recommendations

[0286] Based on the generated diagnostic results, the server generates suggestions for specific fashion items, brands, and stores, and transmits them to the terminal to present to the user.

[0287] Virtual try-on experience using smart glasses

[0288] 1. Capture images of your face and body

[0289] The smart glasses capture images of the user's face and body and send them to a server.

[0290] 2. Providing a virtual try-on experience

[0291] Based on the diagnostic results received from the server, the user is given a virtual fitting experience through the smart glasses, allowing them to try on clothes without actually going to a store.

[0292] Specific examples

[0293] Example 1: A woman in her 30s who likes casual fashion

[0294] 1. User: Puts on smart glasses, accesses the app, and enters age (30s), gender (female), and fashion preference (casual).

[0295] 2. Terminal: Sends input information to the server.

[0296] 3. User: Take a photo of their face with smart glasses and upload it.

[0297] 4. Server: Analyzes the facial image and determines that the user is a "spring type." It then generates a dialogue question, asking, "What color clothes do you like?"

[0298] 5. User: "I like pastel colors."

[0299] 6. Server: Based on the answers, identify pastel-colored casual fashion items and suitable brands and generate a diagnosis.

[0300] 7. Terminal: Displays diagnostic results and provides a virtual try-on experience.

[0301] 8. User: Virtually try on the item and decide to purchase from the suggested store.

[0302] Prompt Sentence Examples

[0303] Users enter their age, gender, fashion preferences, and upload a face image. They then answer the following interactive questions:

[0304] 1. What color clothes do you like?

[0305] 2. Where do you go on your days off?

[0306] 3. What style do you like?

[0307] Based on this information, we will display diagnostic results of recommended fashion items, brands, and shops.

[0308] This allows users to have a realistic try-on experience even remotely, and receive highly accurate fashion suggestions.

[0309] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0310] Step 1:

[0311] Enter profile information

[0312] The user puts on the smart glasses and accesses the application, entering their age, gender, and fashion preferences. This profile information is entered into the device and sent to the server.

[0313] Input: Age, Gender, Fashion Preferences

[0314] Output: Profile information sent to the server

[0315] Step 2:

[0316] Enter and send a facial image

[0317] Users use smart glasses to take a picture of their face and upload it to their device, which then sends the image to a server.

[0318] Input: User's face image

[0319] Output: Face image sent to the server

[0320] Step 3:

[0321] Facial image analysis

[0322] The server performs image analysis on the received facial image. This analysis uses OpenCV and Keras models to extract bone structure and personal color features. The analysis results are temporarily saved.

[0323] Input: Face image

[0324] Output: Bone structure and personal color analysis results

[0325] Step 4:

[0326] Interactive question generation

[0327] The server generates interactive questions based on the user's profile information and facial image analysis results, which are then sent to the device and displayed to the user.

[0328] Input: Profile information, facial image analysis results

[0329] Output: Question to the user

[0330] Step 5:

[0331] Receiving and analyzing user responses

[0332] Users answer questions displayed through their devices, and the answers are sent to a server that analyzes the user's answers, using a generative AI model to interpret the meaning of the answers and dynamically generate the next question.

[0333] Input: User's answer

[0334] Output: Parsed answer, next question

[0335] Step 6:

[0336] Generate comprehensive diagnostic results

[0337] The server integrates the received profile information, facial image analysis results, and the user's responses, and uses a generative AI model to generate a comprehensive diagnostic result, which includes suitable fashion items, brands, and stores for the user.

[0338] Input: Profile information, facial image analysis results, user responses

[0339] Output: Overall diagnostic results

[0340] Step 7:

[0341] Displaying diagnostic results

[0342] The server sends the generated diagnostic results to the device, which displays them to the user and prepares for the virtual try-on experience.

[0343] Input: Overall diagnostic results

[0344] Output: Display diagnostic results to the user

[0345] Step 8:

[0346] Providing a virtual try-on experience

[0347] The smart glasses provide users with a virtual try-on experience based on the diagnostic results received from the server, allowing them to try on clothes in a virtual space without actually going to a store.

[0348] Input: Diagnosis results, face and body images from smart glasses

[0349] Output: Virtual try-on experience

[0350] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0351] MODE FOR CARRYING OUT THE INVENTION

[0352] System program and processing description

[0353] The system of this invention utilizes AI and an emotion engine to help users find fashion and makeup items that suit them. The system operates in cooperation with a server, a terminal, and a user, each of which plays a specific role.

[0354] Server Operation

[0355] 1. Receiving profile information:

[0356] The server receives the user's profile information (age, gender, fashion preferences, etc.) sent from the device and stores it in a database.

[0357] 2. Receiving and analyzing face images:

[0358] The server receives the user's facial image from the device, applies an image analysis algorithm to analyze the user's bone structure and personal color, and temporarily stores the facial image analysis results.

[0359] 3. How the Emotion Engine works:

[0360] The server analyzes the user's emotions in real time based on the received facial images and dialogue content, and recognizes their emotional state, which is used in the next processing step.

[0361] 4. Interactive question generation and answer analysis:

[0362] The server generates interactive questions based on the emotion engine, taking into account the user's momentary emotional state, and asks the questions to the user via the device. It receives the user's answers, analyzes their content, and dynamically generates the next questions to ask. This process is repeated until all necessary information is obtained.

[0363] 5. Generating comprehensive diagnostic results:

[0364] The server integrates the received profile information, facial image analysis results, emotional state, and user responses, and uses an AI model to generate a comprehensive diagnostic result, which includes suitable fashion items, brands, and shops for the user.

[0365] 6. Presentation of results and recommendations:

[0366] Based on the generated diagnostic results, the server generates suggestions for specific fashion items, brands, and shops that take into account the emotional state, and transmits them to the terminal to present to the user.

[0367] Device behavior

[0368] 1. Provide profile information input interface:

[0369] The terminal provides an interface for the user to input profile information and transmits the input information to the server.

[0370] 2. Provide face image upload interface:

[0371] The terminal provides an interface for the user to upload a facial image and transmits the uploaded facial image to the server.

[0372] 3. Providing an interactive question and answer input interface:

[0373] The terminal displays the interactive questions sent from the server, provides an interface for the user to answer the questions, and sends the user's answers to the server.

[0374] 4. Provide diagnostic result display interface:

[0375] The terminal provides an interface for displaying the diagnosis results and suggestions sent from the server to the user.

[0376] User behavior

[0377] 1. Enter your profile information:

[0378] Through the device interface, users enter their profile information, including age, gender, and fashion preferences.

[0379] 2. Upload your face image:

[0380] Users upload their facial images using the device's interface.

[0381] 3. Answer the interactive questions:

[0382] The user answers questions sent from the server through the device. For example, to the question "What color clothes do you like?", the user answers "I like pastel colors."

[0383] 4. Review the diagnostic results and accept the recommendations:

[0384] Users can check the diagnostic results displayed on their device and browse suggested fashion items, brands, and shops.

[0385] Specific examples

[0386] Example 1: A woman in her 30s who likes casual fashion

[0387] 1. User: Accesses the app and enters age (30s), gender (female), and fashion preference (casual).

[0388] 2. Terminal: Sends input information to the server.

[0389] 3. User: Upload a photo of your face.

[0390] 4. Device: Sends a photo of your face to the server.

[0391] 5. Server: Performs facial image analysis and determines the person as a "spring type." The analysis results are temporarily saved.

[0392] 6. Server: Uses an emotion engine to recognize emotions from the user's facial images and answers, and understands their current emotional state.

[0393] 7. Server: Generates questions and asks the user, "What color clothes do you like?". Adjusts the questions depending on the user's emotional state.

[0394] 8. Terminal: displays the question and accepts the user's answer.

[0395] 9. User: "I like pastel colors."

[0396] 10. Terminal: Sends the answer to the server.

[0397] 11. Server: Analyzes the answer and generates the next question. Sends "What kind of clothes would you like to wear for what occasion?". Adjusts the next question based on the emotional state.

[0398] 12. Terminal: Show question.

[0399] 13. User: Enter "Relaxing holiday scene."

[0400] 14. Terminal: Sends the answer to the server, analyzes the answer, and repeats this process until all the necessary information is gathered.

[0401] 15. Server: Using an AI model based on facial image analysis results, answers to interactive questions, and emotional state, the server generates diagnostic results suitable for casual fashion, including information on "pastel-colored items" and "suitable brands."

[0402] 16. Server: Dynamically generates diagnostic results and specific recommendations.

[0403] 17. Terminal: Displays the diagnostic results and suggestions to the user.

[0404] 18. User: Based on the presented results, select a pastel-colored casual fashion item and purchase it from the suggested shop.

[0405] In this way, by combining emotion engines, it is possible to provide highly accurate personalized fashion advice according to the user's emotional state.

[0406] The processing flow will be explained below.

[0407] Program processing steps

[0408] Step 1: Enter your user information

[0409] User: Visits an app or website and enters profile information such as age, gender, and fashion preferences.

[0410] Terminal: Receives profile information entered by the user and sends it to the server.

[0411] Server: Stores the received profile information in a database and prepares for the next step.

[0412] Step 2: Upload and analyze face images

[0413] Users: Upload a photo of themselves to an app or website.

[0414] Terminal: Sends the uploaded face photo to the server.

[0415] Server: Analyzes bone structure and personal color using facial image analysis algorithms. Saves the analysis results temporarily and uses them in the next step.

[0416] Step 3: Emotion analysis using the emotion engine

[0417] Server: Analyzes the user's emotions in real time using an emotion engine based on facial images and dialogue content. Temporarily stores the analysis results.

[0418] Step 4: Interactive question generation and answer analysis

[0419] Server: Generates conversational questions that take into account the user's emotional state and sends them to the device. For example, set questions such as "What color clothes do you like?" or "What kind of clothes do you like to wear for what occasions?"

[0420] Terminal: Presents an interface for displaying interactive questions to the user and entering answers.

[0421] User: Enters an answer to a question.

[0422] Terminal: Sends the entered answer to the server.

[0423] Server: Analyzes the received answers and dynamically generates the next question to ask. This process is repeated until all necessary questions have been asked.

[0424] Step 5: Generate comprehensive diagnostic results

[0425] Server: Integrates the results of facial image analysis, answers collected through dialogue questions, and emotional state, and uses AI models to generate comprehensive diagnostic results. This identifies the user's personal color, clothing styles and designs that suit them, popular brands, and suitable accessories and makeup items.

[0426] Server: Stores the diagnostic results in a database and prepares the results for display.

[0427] Step 6: Present your results and make recommendations

[0428] Server: Based on the diagnosis results, it generates recommendations for specific fashion items, brands, and shops and sends them to the device. The recommendations may be updated in real time.

[0429] Terminal: Provides an interface for displaying the diagnostic results and suggestions sent from the server to the user.

[0430] User: Based on the results presented, users can choose fashion and makeup items that suit them best.

[0431] Specific examples

[0432] Example: A woman in her 30s who likes casual fashion

[0433] 1. Step 1:

[0434] User: Accesses the app and enters age (30s), gender (female), and fashion preference (casual).

[0435] Terminal: Sends input information to the server.

[0436] Server: Saves the input information in a database.

[0437] 2. Step 2:

[0438] User: Upload a photo of themselves to the app.

[0439] Device: Sends a photo of your face to the server.

[0440] Server: Performs facial image analysis and determines "spring type." Temporarily saves the analysis results.

[0441] 3. Step 3:

[0442] Server: Uses an emotion engine to recognize emotions from the user's facial image and responses, and understands their current emotional state. For example, it determines whether the user is in a "peace of mind" state.

[0443] 4. Step 4:

[0444] Server: Based on the emotional state, generate a question like "What color clothes do you like?" and send it to the user. The question content is adjusted depending on the emotional state.

[0445] Terminal: Provides an interface for displaying questions and accepting user answers.

[0446] User: "I like pastel colors."

[0447] Terminal: Sends the answer to the server.

[0448] Server: Analyzes the answer and generates the next question: "What kind of occasions do you want to wear your clothes for?" The next question is adjusted taking into account the emotional state.

[0449] Terminal: Display the question.

[0450] User: Enter "Relaxing holiday scene."

[0451] Device: Sends the answer to the server and repeats this process until it has gathered the required information.

[0452] 5. Step 5:

[0453] Server: Using an AI model based on facial image analysis, responses to interactive questions, and emotional state, the server generates a comprehensive diagnosis suitable for casual fashion, including information on "pastel-colored items" and "suitable brands."

[0454] Server: Stores the diagnostic results in a database.

[0455] 6. Step 6:

[0456] Server: Generates diagnostic results and specific fashion item, brand, and shop recommendations based on the results. Suggestions may be updated in real time.

[0457] Terminal: Provides an interface for displaying diagnostic results and recommendations to the user.

[0458] User: Based on the presented results, select a pastel-colored casual fashion item and purchase it from the suggested shop.

[0459] In this way, users can receive highly accurate and realistic fashion advice that takes their emotions into consideration.

[0460] Example 2

[0461] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0462] Conventional fashion advice systems often make suggestions based solely on the user's profile information or image analysis results, and are unable to consider the user's emotional state or real-time changes in preferences. This makes it difficult to provide personalized, highly accurate fashion advice, resulting in reduced user satisfaction. The present invention aims to solve these problems and provide personalized fashion advice that takes the user's emotional state into account.

[0463] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0464] In this invention, the server includes a means for receiving input information from a user, a means for receiving a facial image of the user and analyzing facial features and color characteristics through image processing, and a means for analyzing the user's emotions in real time and dynamically generating questions based on the analysis results. This makes it possible to integrate the user's profile information, facial image analysis results, emotion analysis results, and user answers, use a generative AI model to generate highly accurate comprehensive diagnostic results, and suggest clothing items, brands, and sales locations that are suitable for the user.

[0465] "Input Information" means the personal information and profile data that a User provides to the System.

[0466] A "face image" refers to a photograph or image data of a user's face.

[0467] "Image processing" refers to the technology of analyzing image data and extracting specific features or attributes.

[0468] "Facial features" refers to the shape, bone structure, and feature points of the user's face.

[0469] "Color characteristics" refers to a personal color classification based on the user's face and skin color.

[0470] "Emotion analysis" is a technology that analyzes a user's emotional state in real time from facial images and dialogue content.

[0471] A "generative AI model" is an artificial intelligence model that uses machine learning and natural language processing to analyze data and generate interactive questions and diagnostic results.

[0472] "Comprehensive diagnosis results" are diagnostic results generated by the AI ​​model based on the user's profile information, facial image analysis results, emotion analysis results, and the user's responses.

[0473] "Clothing items" refers to clothes, accessories, etc. suggested to users.

[0474] "Trademark" means the name or logo of a particular brand or manufacturer, which is associated with the proposed fashion item.

[0475] "Point of Sale" refers to a store or online shop where the products suggested to the user can be purchased.

[0476] This invention is a system that utilizes AI and an emotion engine to help users find fashion and makeup items that suit them. The system works in cooperation with a server, a device, and a user, each of which plays a specific role.

[0477] Server Operation

[0478] 1. Receiving profile information

[0479] The server receives the user's input information (age, gender, fashion preferences, etc.) sent from the device and stores it in a database. Specifically, it uses a MySQL database.

[0480] 2. Receiving and analyzing face images

[0481] After receiving the user's facial image from the device, the server processes the image using OpenCV and Dlib. The facial image is analyzed to analyze the user's skeletal and color characteristics, and the analysis results are temporarily stored in Redis.

[0482] 3. Operation of the Emotion Engine

[0483] The server uses Microsoft Azure's emotion analysis API to analyze the user's emotions in real time from facial images and dialogue content, recognizing their emotional state. The analysis results are used in the next step of question generation.

[0484] 4. Question Generation and Answer Analysis

[0485] The server generates questions that take into account the user's temporary emotional state based on the results of sentiment analysis. The generated questions are sent to the user via the device and the user's answers are received. The received answers are then analyzed and the next question is dynamically generated. The NLP technology used is SpaCy and the BERT model.

[0486] 5. Generating comprehensive diagnostic results

[0487] The server integrates the received input information, facial image analysis results, emotional state, and user responses, and uses a generative AI model (e.g., GPT-4) to generate a comprehensive diagnosis result, which includes clothing items, brands, and sales locations suitable for the user.

[0488] 6. Presentation of results and recommendations

[0489] Based on the generated diagnostic results, the server generates specific suggestions that take into account the emotional state, and transmits them to the terminal to present to the user.

[0490] Device behavior

[0491] 1. Provides a profile information input interface

[0492] The terminal provides an interface for users to input information and sends the input information to the server, specifically using JavaScript and ReactJS.

[0493] 2. Providing a face image upload interface

[0494] The terminal provides an interface for the user to upload a facial image and transmits the uploaded facial image to the server.

[0495] 3. Providing a question and answer input interface

[0496] The terminal provides an interface for displaying the questions sent from the server and accepting the user's answers, and then sends the user's answers to the server.

[0497] 4. Providing an interface for displaying diagnostic results

[0498] The terminal provides an interface for displaying the diagnosis results and suggestions sent from the server to the user.

[0499] User behavior

[0500] 1. Enter your profile information

[0501] Through the device's interface, users enter their personal information, including age, gender, and fashion preferences.

[0502] 2. Upload a face image

[0503] Users upload their facial images using the device's interface.

[0504] 3. Answering questions

[0505] The user answers questions sent from the server through the device. For example, to the question "What color clothes do you like?", the user answers "I like pastel colors."

[0506] 4. Review the diagnostic results and accept the recommendations

[0507] The user checks the diagnostic results displayed on the device and browses suggested clothing items, brands, and sales locations.

[0508] Specific examples

[0509] Example 1: A woman in her 30s who likes casual fashion

[0510] 1. User: Accesses the app and enters age (30s), gender (female), and fashion preference (casual).

[0511] 2. Terminal: Sends input information to the server.

[0512] 3. Server: Stores profile information in a MySQL database.

[0513] 4. User: Upload a photo of your face.

[0514] 5. Device: Sends a photo of your face to the server.

[0515] 6. Server: Performs facial image analysis and determines "spring type." Temporarily stores the analysis results in Redis.

[0516] 7. Server: Uses an emotion engine to recognize emotions from the user's facial images and answers, and understands their current emotional state.

[0517] 8. Server: Generates questions and asks the user, "What color clothes do you like?". Adjusts the questions depending on the user's emotional state.

[0518] 9. Terminal: displays the question and accepts the user's answer.

[0519] 10. User: "I like pastel colors."

[0520] 11. Terminal: Sends the answer to the server.

[0521] 12. Server: Analyzes the answer and generates the next question. Sends "What kind of clothes would you like to wear for what occasion?". Adjusts the next question based on the emotional state.

[0522] 13. Terminal: Display the question.

[0523] 14. User: Enter "Relaxing holiday scene."

[0524] 15. Terminal: Sends the answer to the server, analyzes the answer, and repeats this process until all the necessary information is gathered.

[0525] 16. Server: Using a generative AI model based on facial image analysis results, answers to interactive questions, and emotional state, the server generates diagnostic results suitable for casual fashion, including information on "pastel-colored items" and "suitable brands."

[0526] 17. Server: Dynamically generates diagnostic results and specific recommendations.

[0527] 18. Terminal: Display the diagnostic results and suggestions to the user.

[0528] 19. User: Based on the presented results, select a pastel-colored casual fashion item and purchase it at the suggested retail location.

[0529] Prompt Sentence Examples

[0530] "I'm a woman in my 30s who likes casual fashion. My face image indicates I'm a spring type. What fashion items and brands would you suggest?"

[0531] In this way, it is possible to provide personalized fashion advice according to the user's emotional state.

[0532] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0533] Step 1:

[0534] Enter profile information

[0535] The user enters their own information (age, gender, fashion preferences, etc.) through the device interface. The device validates the entered information using JavaScript and sends it to the server. The server receives the input information and saves it in a MySQL database using INSERT statements. The input is the user's profile information, and the output is the profile information saved in the database.

[0536] Step 2:

[0537] Upload a face image

[0538] A user uploads their facial image using the device interface. The device encodes the uploaded facial image into Base64 format and sends it to the server. The server receives the facial image and temporarily stores it in a file system or database. The input is the user's facial image, and the output is the facial image stored on the server.

[0539] Step 3:

[0540] Facial image analysis

[0541] The server analyzes the stored facial images using OpenCV and Dlib. First, it uses a face detection algorithm to extract facial feature points, then analyzes the bone structure and color characteristics. The analysis results are temporarily stored in Redis. The input is the user's facial image, and the output is the analysis results including bone structure and color characteristics.

[0542] Step 4:

[0543] Emotion analysis

[0544] The server uses Microsoft Azure's emotion analysis API to analyze the user's emotions in real time from facial images and dialogue content. It makes API calls, obtains the analysis results, and stores them in a database. The input is the user's facial image and dialogue content, and the output is the analysis results that indicate the user's emotional state.

[0545] Step 5:

[0546] Question generation and answer analysis

[0547] The server takes into account the user's emotional state based on the sentiment analysis results and uses a generative AI model to dynamically generate the next question. It then sends the generated question to the user via their device and receives the user's answer. It analyzes the answer and, if necessary, generates the next question and repeats the process. The input is the sentiment analysis result and the user's answer, and the output is the dynamically generated next question.

[0548] Step 6:

[0549] Generate comprehensive diagnostic results

[0550] The server integrates the profile information, facial image analysis results, emotion analysis results, and the user's responses, and uses a generative AI model to generate a comprehensive diagnosis. Specifically, a prompt is entered to query the AI ​​model and obtain the results. The diagnosis results include clothing items, trademarks, and sales locations suitable for the user. The input is the integrated user information, and the output is the comprehensive diagnosis results.

[0551] Step 7:

[0552] Presentation of results and recommendations

[0553] The server sends the generated diagnostic results to the terminal. The terminal provides an interface to display the diagnostic results and suggestions to the user. The user can review the presented results and select items that suit their preferences. The input is the overall diagnostic results, and the output is the suggestions displayed to the user.

[0554] (Application example 2)

[0555] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0556] Conventional fashion and makeup item recommendation systems have issues with insufficient suggestions based on user profile information or facial image analysis, and are unable to provide personalized advice that takes into account the user's emotional state. Furthermore, they lack the ability to dynamically generate questions that respond to the user's preferences and emotions, or to adjust the recommendations in real time, making it difficult to recommend optimal products to users.

[0557] The identification processing by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for receiving profile information from a user, means for receiving a facial image of the user and analyzing the user's bone structure and personal color through image analysis, means for interactively asking the user questions and analyzing the user's responses, means for integrating the received profile information, facial image analysis results, and the user's responses and generating a comprehensive diagnostic result using an AI model, means for proposing products suitable for the user based on the diagnostic result, means for analyzing the user's emotional state in real time and dynamically adjusting the content of the interactive questions based on the emotional state, and means for inputting prompt sentences into a generative AI model generated based on the emotional state and profile information to make personalized suggestions. This enables optimal product suggestions in real time while taking the user's emotional state into consideration.

[0558] "Profile Information" is information about a user's personal information, such as their age, gender, and fashion preferences.

[0559] "Bone structure" refers to features that represent the structure of a user's face and body.

[0560] "Personal Color" refers to the range of colors that best complement a user's natural skin tone and hair color.

[0561] "Dialogue" is a format in which the user and the system collect information by exchanging questions and answers.

[0562] An "AI model" is a mathematical model that uses machine learning and artificial intelligence techniques to analyze data and make recommendations and predictions.

[0563] The "comprehensive diagnosis result" is the result of optimal suggestions for the user, generated based on the received profile information, facial image analysis results, and user responses.

[0564] "Emotional state" indicates the user's current emotional state and is estimated from facial expressions and dialogue content.

[0565] A "generative AI model" is an AI model that generates personalized suggestions based on a user's profile information and emotional state using specific prompt sentences as input.

[0566] A "prompt sentence" is text that is input into a generative AI model and serves as the basis for generating specific suggestions for the user.

[0567] "Dynamic adjustment" means making changes and adaptations instantly in response to the user's real-time reactions and conditions.

[0568] This invention relates to a system that enables users to find fashion items and makeup items that suit them, and is mainly comprised of a server, a terminal, and a user, each of which plays a specific role and operates in cooperation with one another.

[0569] System configuration

[0570] Server Operation

[0571] 1. The server receives the user's profile information (age, gender, fashion preferences, etc.) sent from the device and stores it in a database.

[0572] 2. After receiving the user's facial image from the device, the server applies image analysis algorithms to analyze the facial structure and personal color. This image analysis uses image processing libraries such as OpenCV.

[0573] 3. The server uses the Emotion Engine to analyze the user's emotions in real time based on the received facial images and dialogue content, and recognizes their emotional state.

[0574] 4. The server generates dialogue-style questions based on the emotion engine, taking into account the user's temporary emotional state, and asks the questions to the user via the terminal. This can be done using a framework such as Flask or Django.

[0575] 5. The server integrates the received profile information, facial image analysis results, emotional state, and the user's answers, and generates a comprehensive diagnosis result using an AI model called AIFashionModel, which includes fashion items, brands, and shops suitable for the user.

[0576] 6. The server generates suggestions for specific fashion items, brands, and shops that take into account the emotional state, and sends them to the terminal to present to the user.

[0577] Device behavior

[0578] 1. The terminal provides an interface for the user to input profile information and transmits the input information to the server.

[0579] 2. The terminal provides an interface for the user to upload a facial image and transmits the uploaded facial image to the server.

[0580] 3. The terminal displays the interactive questions sent from the server, provides an interface for the user to answer the questions, and sends the user's answers to the server.

[0581] 4. The terminal provides an interface for displaying the diagnostic results and suggestions sent from the server to the user.

[0582] User behavior

[0583] 1. Through the device interface, users enter their profile information, including age, gender, and fashion preferences.

[0584] 2. The user uploads an image of their face using the device interface.

[0585] 3. The user answers questions sent from the server through their device. For example, to the question "What color clothes do you like?", the user answers "I like pastel colors."

[0586] 4. The user checks the diagnostic results displayed on the device and browses suggested fashion items, brands, and shops.

[0587] Specific examples

[0588] For example, for a female user in her 30s who likes casual fashion,

[0589] 1. The user enters their age (30s), gender (female), and fashion preference (casual).

[0590] 2. Upload a photo of your face and your face will be analyzed to determine your "spring type."

[0591] 3. The server uses an emotion engine to recognize the user's "current emotional state" from the facial image and generates interactive questions.

[0592] 4. For example, ask the user, "What color clothes do you like?"

[0593] 5. The user answers, "I like pastel colors," and based on this, the next question is dynamically generated, such as, "What kind of clothes do you like to wear for what occasions?"

[0594] 6. This process is repeated until all the necessary information is gathered, and finally, personalized fashion items are suggested.

[0595] Prompt Sentence Examples

[0596] If the profile information is "30s, female, casual fashion" and the emotional state is "happy," the server will input the following prompt sentence to the AI ​​model:

[0597] "A woman in her 30s likes casual fashion and spring-type pastel colors. She is currently in a happy emotional state. Please suggest some fashion items that would be perfect for relaxing on the weekend."

[0598] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0599] Step 1:

[0600] Users enter profile information through the device interface, such as age, gender, and fashion preferences. This information becomes input data, which the device receives and transmits to the server.

[0601] Step 2:

[0602] The server stores the user profile information received from the device in a database, which will be referenced in subsequent processing.

[0603] Step 3:

[0604] The user takes a picture of their face using the device's camera and uploads it to the server. The face image becomes input data, and the device receives this face image data and sends it to the server.

[0605] Step 4:

[0606] The server analyzes the received facial images using an image processing library such as OpenCV. This analysis identifies the facial bone structure and personal color, which are then temporarily saved as intermediate data.

[0607] Step 5:

[0608] The server uses EmotionEngine to analyze the user's emotional state in real time based on the received facial images and dialogue content. The emotional state is output as an analysis result and used in the next step.

[0609] Step 6:

[0610] Based on the analyzed emotional state, the server dynamically generates interactive questions, which are tailored to the user's current emotions and sent to the device as question data.

[0611] Step 7:

[0612] The terminal presents the user with interactive questions sent from the server, and the user answers them, and the terminal receives the answer data and sends it to the server.

[0613] Step 8:

[0614] The server analyzes the user's answers and dynamically generates subsequent questions to obtain further required information. This process is repeated until the required information is gathered.

[0615] Step 9:

[0616] The server integrates the received profile information, facial image analysis results, emotional state, and user responses, and generates a comprehensive diagnosis result using the AIFashionModel, which includes a list of products suitable for the user, and generates this data as recommendation data.

[0617] Step 10:

[0618] The server takes into account the emotional state and diagnosis results and inputs the optimal suggestion into the generative AI model as a prompt sentence, which contains specific fashion item suggestions for the user.

[0619] Step 11:

[0620] The generated suggestions are sent to the device and presented to the user. The user can use these suggestions as a reference when selecting products. For example, a prompt might read, "A woman in her 30s likes casual fashion and spring-type pastel colors. Her current emotional state is happy. Please suggest the best fashion items for a relaxing holiday."

[0621] This makes it possible to provide personalized fashion advice and suggestions that take into account emotional state in real time.

[0622] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0623] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0624] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0625] [Second embodiment]

[0626] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0627] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0628] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0629] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0630] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0631] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0632] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0633] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0634] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0635] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0636] In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0637] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0638] MODE FOR CARRYING OUT THE INVENTION

[0639] System program and processing description

[0640] The system of this invention utilizes AI to help users find fashion and makeup items that suit them. The system operates in cooperation with a server, a terminal, and a user, each of which plays a specific role.

[0641] Server Operation

[0642] 1. Receiving profile information:

[0643] The server receives the user's profile information (age, gender, fashion preferences, etc.) sent from the device and stores it in a database.

[0644] 2. Receiving and analyzing face images:

[0645] The server receives the user's facial image from the device, applies an image analysis algorithm to analyze the user's bone structure and personal color, and temporarily stores the facial image analysis results.

[0646] 3. Interactive question generation and answer analysis:

[0647] The server generates interactive questions and asks them to the user via the terminal, receives the user's answers, analyzes the answers, and dynamically generates the next questions to ask. This process is repeated until all necessary information is obtained.

[0648] 4. Generating comprehensive diagnostic results:

[0649] The server integrates the received profile information, facial image analysis results, and user responses, and uses an AI model to generate a comprehensive diagnosis, which includes suitable fashion items, brands, and shops for the user.

[0650] 5. Presentation of results and recommendations:

[0651] Based on the generated diagnostic results, the server generates suggestions for specific fashion items, brands, and shops, and transmits them to the terminal to present to the user.

[0652] Device behavior

[0653] 1. Provide profile information input interface:

[0654] The terminal provides an interface for the user to input profile information and transmits the input information to the server.

[0655] 2. Provide face image upload interface:

[0656] The terminal provides an interface for the user to upload a facial image and transmits the uploaded facial image to the server.

[0657] 3. Providing an interactive question and answer input interface:

[0658] The terminal displays the questions sent from the server, provides an interface for the user to answer the questions, and sends the user's answers to the server.

[0659] 4. Provide diagnostic result display interface:

[0660] The terminal provides an interface for displaying the diagnosis results and suggestions sent from the server to the user.

[0661] User behavior

[0662] 1. Enter your profile information:

[0663] Through the device interface, users enter their profile information, including age, gender, and fashion preferences.

[0664] 2. Upload your face image:

[0665] Users upload their facial images using the device's interface.

[0666] 3. Answer the interactive questions:

[0667] The user answers questions sent from the server through the device. For example, to the question "What color clothes do you like?", the user answers "I like pastel colors."

[0668] 4. Review the diagnostic results and accept the recommendations:

[0669] Users can check the diagnostic results displayed on their device and browse suggested fashion items, brands, and shops.

[0670] Specific examples

[0671] Example 1: A woman in her 30s who likes casual fashion

[0672] 1. User: Accesses the app and enters age (30s), gender (female), and fashion preference (casual).

[0673] 2. Terminal: Sends input information to the server.

[0674] 3. User: Upload a photo of your face.

[0675] 4. Device: Send a photo of your face to the server.

[0676] 5. Server: Performs facial image analysis and determines the user is a "spring type." Generates a dialogue question, asking, "What color clothes do you like?"

[0677] 6. User: "I like pastel colors."

[0678] 7. Terminal: Sends the answer to the server.

[0679] 8. Server: Based on all the information, identify pastel-colored casual fashion items and suitable brands and generate a diagnosis.

[0680] 9. Terminal: Display the diagnostic results.

[0681] 10. User: Select an item based on the diagnosis results and purchase it from the suggested shop.

[0682] Example 2: A man in his 40s who likes formal fashion

[0683] 1. User: Accesses the app and enters age (40s), gender (male), and fashion preference (formal).

[0684] 2. Terminal: Sends input information to the server.

[0685] 3. User: Upload a photo of your face.

[0686] 4. Device: Send a photo of your face to the server.

[0687] 5. Server: Performs facial image analysis and determines the person is a "winter type." Generates a dialogue question, asking, "What color do you prefer for formal occasions?"

[0688] 6. User: "Dark blue or black."

[0689] 7. Terminal: Sends the answer to the server.

[0690] 8. Server: Based on all the information, it identifies formal fashion items and popular brands and generates diagnostic results.

[0691] 9. Terminal: Display the diagnostic results.

[0692] 10. User: Select an item based on the diagnosis results and purchase it from the suggested shop.

[0693] In this way, users can easily find the fashion style that suits them through highly accurate diagnosis. The system handles everything from entering profile information, analyzing facial images, asking interactive questions, and generating and proposing comprehensive diagnosis results.

[0694] The processing flow will be explained below.

[0695] Program processing steps

[0696] Step 1: Enter your user information

[0697] User: Visits an app or website and enters profile information such as age, gender, and fashion preferences.

[0698] Terminal: Receives profile information entered by the user and sends it to the server.

[0699] Server: Stores the received profile information in a database and prepares for the next step.

[0700] Step 2: Upload and analyze face images

[0701] Users: Upload a photo of themselves to an app or website.

[0702] Terminal: Sends the uploaded face photo to the server.

[0703] Server: Analyzes bone structure and personal color using facial image analysis algorithms. Saves the analysis results temporarily and uses them in the next step.

[0704] Step 3: Interactive question generation and answer analysis

[0705] Server: Generates interactive questions and sends them to the device. Initial questions include "What color clothes do you like?" and "What kind of clothes do you like to wear for what occasions?"

[0706] Terminal: Provides an interface for displaying interactive questions to the user and for entering answers.

[0707] User: Enters an answer to a question.

[0708] Terminal: Sends the entered answer to the server.

[0709] Server: Analyzes the received answers and dynamically generates the next question to ask. This process is repeated until all necessary questions have been asked.

[0710] Step 4: Generate comprehensive diagnostic results

[0711] Server: The server combines the results of facial image analysis with the user's responses to interactive questions and uses an AI model to generate a comprehensive diagnosis, specifically identifying the user's personal color, clothing styles and designs that suit them, popular brands, and suitable accessories and makeup items.

[0712] Server: Stores the diagnostic results in a database and prepares the results for display.

[0713] Step 5: Present the results and make recommendations

[0714] Server: Generates the diagnosis results and specific fashion item, brand, and shop recommendations based on them. The recommendations may be updated in real time, so they must be generated dynamically.

[0715] Terminal: Provides an interface for presenting diagnostic results to the user.

[0716] User: Based on the results presented, users can choose fashion and makeup items that suit them best.

[0717] Specific examples

[0718] Example: A woman in her 30s who likes casual fashion

[0719] 1. Step 1:

[0720] User: Accesses the app and enters age (30s), gender (female), and fashion preference (casual).

[0721] Terminal: Sends input information to the server.

[0722] Server: Saves the input information in a database.

[0723] 2. Step 2:

[0724] User: Upload a photo of themselves to the app.

[0725] Device: Sends a photo of your face to the server.

[0726] Server: Performs facial image analysis and determines "spring type." Temporarily saves the analysis results.

[0727] 3. Step 3:

[0728] Server: Generates a dialogue question and asks the user, "What color clothes do you like?"

[0729] Terminal: Displays questions and accepts user answers.

[0730] User: "I like pastel colors."

[0731] Terminal: Sends the answer to the server.

[0732] Server: Analyzes the answer and generates the next question: "What kind of clothes would you like to wear for what occasion?"

[0733] Terminal: Show question.

[0734] User: Enter "Relaxing holiday scene."

[0735] Device: Sends the answer to the server, analyzes the answer, and repeats this process until it has gathered all the information it needs.

[0736] 4. Step 4:

[0737] Server: Using an AI model based on the results of facial image analysis and answers to interactive questions, the server generates diagnostic results suitable for casual fashion, including information on "pastel-colored items" and "suitable brands."

[0738] Server: Stores the diagnostic results in a database.

[0739] 5. Step 5:

[0740] Server: Dynamically generates diagnostic results and recommendations for specific fashion items, brands, and shops based on those results.

[0741] On the device: Display diagnostic results and suggestions to the user.

[0742] User: Based on the presented results, select a pastel-colored casual fashion item and purchase it from the suggested shop.

[0743] In this way, users can receive highly accurate and realistic fashion advice throughout the process.

[0744] Example 1

[0745] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0746] Users often spend a lot of time and effort finding the perfect fashion and makeup items for themselves, and it can be particularly difficult to choose items that take into account facial features and personal color. Current systems are unable to accurately analyze these points and make real-time suggestions that match the user's preferences and characteristics. This leaves users with the challenge of being unable to quickly and easily find the perfect fashion and makeup items based on their preferences and characteristics.

[0747] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0748] In this invention, the server includes means for receiving profile information from a user, means for receiving a facial image of the user and analyzing bone structure and personal color through image analysis, means for interactively asking the user questions and analyzing the user's responses, means for integrating the received profile information, facial image analysis results, and user responses and generating a comprehensive diagnostic result using an artificial intelligence model, means for suggesting clothing items, brands, and retailers suitable for the user based on the diagnostic result, communication means for sending initial information to the server in JSON format, means for sending a facial image via an HTTP request, means for analyzing the facial image using an image analysis library, means for dynamically generating interactive questions using the profile information and image analysis results, and means for generating and presenting a comprehensive diagnostic result, thereby enabling users to quickly and easily find fashion and makeup items based on their characteristics and preferences.

[0749] "Profile information" refers to information about a user's personal characteristics, such as the user's age, gender, and fashion preferences.

[0750] A "face image" is an image of the user's face, and facial features and personal color are analyzed based on this image.

[0751] "Image analysis" is a process performed on a received facial image to identify the user's bone structure and personal color.

[0752] "Dialogue format" refers to a format in which questions are asked to the user sequentially, and the next question is dynamically generated based on the answers.

[0753] An "artificial intelligence model" is a model that uses technologies such as machine learning and deep learning, and generates comprehensive diagnostic results based on the received data.

[0754] The "comprehensive diagnosis results" are generated by integrating profile information, facial image analysis results, and user responses, and include suggestions for fashion items, trademarks, and retailers that are suitable for the user.

[0755] "Communication means" refers to the technical means for transmitting initial information and analysis data to the server in JSON format or via HTTP requests.

[0756] The "JSON format" is a format that structures and expresses data in text format, and is primarily used for sending and receiving data.

[0757] An "HTTP request" is a protocol that allows a client to request a server to send data or obtain information.

[0758] An "image analysis library" is a software library, such as OpenCV or Dlib, that is used to extract and analyze features from facial images.

[0759] "Dynamic generation" means adaptively generating the next question or suggestion on the spot based on the user's answers.

[0760] "Diagnosis results" are recommendations for fashion items and services suitable for the user, generated based on analysis of the received data and artificial intelligence models.

[0761] MODE FOR CARRYING OUT THE INVENTION

[0762] This invention provides a system in which a server, a terminal, and a user work together to help users find the fashion and makeup items that are best suited to them. The overall configuration of the system and the operation of each component are described in detail below.

[0763] Server Operation

[0764] 1. Receiving and storing profile information:

[0765] The server receives the user's profile information (age, gender, fashion preferences, etc.) sent from the device and stores it in a database (e.g., MySQL or PostgreSQL), making it possible to manage each user's individual information.

[0766] 2. Receiving and analyzing face images:

[0767] After receiving the user's facial image from the device, the server analyzes the user's bone structure and personal color using image analysis libraries such as OpenCV and Dlib. The analysis results are temporarily stored in a database.

[0768] 3. Interactive question generation and answer analysis:

[0769] The server uses an NLP library (e.g., NLTK or SpaCy) to generate interactive questions and pose them to the user through the terminal. After receiving the user's answers, it analyzes their content and dynamically generates the next questions.

[0770] 4. Generating comprehensive diagnostic results:

[0771] The server integrates the profile information, facial image analysis results, and user responses, and generates a comprehensive diagnostic result using an AI model (e.g., TensorFlow or PyTorch), which can identify suitable fashion and makeup items for the user.

[0772] 5. Presentation of results and recommendations:

[0773] The server generates specific suggestions based on the generated diagnostic results and sends them to the terminal for presentation to the user, including clothing items, brands, and retailers suitable for the user.

[0774] Device behavior

[0775] 1. Provide profile information input interface:

[0776] The terminal provides an interface for the user to input profile information and transmits the input information to the server.

[0777] 2. Provide face image upload interface:

[0778] The terminal provides an interface for the user to upload a facial image and transmits the uploaded facial image to the server.

[0779] 3. Providing an interactive question and answer input interface:

[0780] The terminal displays the questions sent from the server, provides an interface for the user to answer the questions, and sends the user's answers to the server.

[0781] 4. Provide diagnostic result display interface:

[0782] The terminal provides an interface for displaying the diagnosis results and suggestions sent from the server to the user.

[0783] User behavior

[0784] 1. Enter your profile information:

[0785] Users enter their profile information through the device interface, including their age, gender and fashion preferences.

[0786] 2. Upload your face image:

[0787] Users upload their facial images using the device's interface.

[0788] 3. Answer the interactive questions:

[0789] The user answers questions sent from the server through the device. For example, to the question "What color clothes do you like?", the user answers "I like pastel colors."

[0790] 4. Review the diagnostic results and accept the recommendations:

[0791] Users can check the diagnosis results displayed on their device and use the suggested fashion and makeup items as a reference.

[0792] Specific use cases

[0793] Example 1: A woman in her 30s who likes casual fashion

[0794] 1. User: Accesses the app and enters age (30s), gender (female), and fashion preference (casual).

[0795] 2. Terminal: Sends input information to the server.

[0796] 3. User: Upload a photo of yourself.

[0797] 4. Device: Sends a photo of your face to the server.

[0798] 5. Server: Performs facial image analysis and determines the user is a "spring type." Generates a dialogue question, asking, "What color clothes do you like?"

[0799] 6. User: "I like pastel colors."

[0800] 7. Terminal: Sends the answer to the server.

[0801] 8. Server: Based on all the information, it identifies pastel-colored casual fashion items and suitable trademarks and generates a diagnosis result.

[0802] 9. Terminal: Display the diagnostic results.

[0803] 10. User: Select an item based on the diagnosis results and purchase it from the suggested retailer.

[0804] Example 2: A man in his 40s who likes formal fashion

[0805] 1. User: Accesses the app and enters age (40s), gender (male), and fashion preference (formal).

[0806] 2. Terminal: Sends input information to the server.

[0807] 3. User: Upload a photo of yourself.

[0808] 4. Device: Sends a photo of your face to the server.

[0809] 5. Server: Performs facial image analysis and determines the person is a "winter type." Generates a dialogue question, asking, "What color do you prefer for formal occasions?"

[0810] 6. User: "I like dark blue and black."

[0811] 7. Terminal: Sends the answer to the server.

[0812] 8. Server: Based on all the information, it identifies formal fashion items and popular brands and generates diagnostic results.

[0813] 9. Terminal: Display the diagnostic results.

[0814] 10. User: Select an item based on the diagnosis results and purchase it from the suggested retailer.

[0815] In this way, users can quickly and easily find the fashion and makeup items that best suit them through the system.

[0816] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0817] Step 1: Enter your profile information

[0818] User Action: The user enters their profile information (age, gender, fashion preferences) into the interface provided by the device.

[0819] Input: User profile information (e.g., "30s," "Female," "Casual")

[0820] Output: Sending profile information from device to server

[0821] Specific action: Enter data into a form in a web application and press the "Submit" button.

[0822] Step 2: Submit your profile information

[0823] Device behavior: The device sends the entered profile information to the server in JSON format.

[0824] Input: Profile information entered by the user

[0825] Output: Profile information received by the server

[0826] What it does: Sends profile information to the / api / profile endpoint using an HTTP POST request.

[0827] Step 3: Upload your face image

[0828] User action: The user selects and uploads a face image using the device interface.

[0829] Input: A face image selected by the user

[0830] Output: Sending face image from device to server

[0831] Specific operation: Select an image file and press the "Upload" button.

[0832] Step 4: Send a face image

[0833] Device operation: The device sends the uploaded facial image to the server.

[0834] Input: Face image file uploaded by the user

[0835] Output: Face image received by the server

[0836] What it does: Sends an image file to the / api / upload endpoint using an HTTP POST request.

[0837] Step 5: Facial image analysis

[0838] Server operation: The server analyzes the received facial image using an image analysis library such as OpenCV or Dlib to identify bone structure and personal color.

[0839] Input: Face image sent to the server

[0840] Output: Analysis results of bone structure and personal color

[0841] Specific operation: Run the Python script and use the analyze_face(image) function to extract and identify features from the facial image.

[0842] Step 6: Interactive question generation and answer collection

[0843] Server operation: The server uses an NLP library to generate interactive questions based on the collected information and sends the questions to the user via the terminal.

[0844] Terminal operation: The terminal displays questions sent from the server and sends answers from the user to the server.

[0845] User action: The user answers questions sent by the server through the terminal.

[0846] Input: Server-generated question, user-entered answer

[0847] Output: The user's answer received by the server

[0848] Specific action: In response to the question "What color clothes do you like?", enter "I like pastel colors" as the answer and submit.

[0849] Step 7: Generate comprehensive diagnostic results

[0850] Server operation: The server integrates the user's profile information, facial image analysis results, and answers to interactive questions, and uses an AI model to generate a comprehensive diagnosis result.

[0851] Input: User profile information, facial image analysis results, answers to dialogue questions

[0852] Output: Comprehensive diagnostic results (suitable fashion items, brands, and retailers)

[0853] Specific operation: Using the TensorFlow model, generate diagnostic results with the generate_recommendations(profile, face_analysis, responses) function.

[0854] Step 8: Send and view results

[0855] Server operation: The server sends the generated diagnostic results to the terminal.

[0856] Device behavior: The device displays the received diagnostic results to the user.

[0857] User Action: User reviews diagnostic results and views suggested items.

[0858] Input: Server-generated diagnostic results

[0859] Output: Diagnostics displayed on the terminal

[0860] How it works: The diagnosis results are displayed on the device screen, and the user can check out the suggested fashion and makeup items.

[0861] In this way, users can quickly and easily find the fashion and makeup items that best suit them through the system.

[0862] (Application example 1)

[0863] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0864] With so many options available, it can be difficult for users to find the right fashion or makeup items. There is also a need for the ability to try on items without actually going to a store. However, the current lack of appropriate technology to make this a reality is problematic. Furthermore, there is a growing need for systems that can dynamically reflect users' preferences and characteristics in real time and make highly accurate recommendations.

[0865] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0866] In this invention, the server includes means for receiving profile information from a user, means for receiving a facial image of the user and analyzing the image to analyze bone structure and personal color, means for interactively asking the user questions and analyzing the user's responses, means for integrating the received profile information, facial image analysis results, and user responses and generating a comprehensive diagnostic result using an AI model, means for suggesting fashion items, brands, and stores suitable for the user based on the generated diagnostic result, and means for capturing images of the user's face and body using smart glasses to provide a virtual try-on experience, allowing the user to try on fashion items in a realistic way without actually going to a store and receiving highly accurate suggestions for fashion items.

[0867] "Profile information" refers to information such as a user's age, gender, and fashion preferences.

[0868] "Facial Image" refers to a photograph or video of a user's face.

[0869] "Image analysis" refers to the process of analyzing received facial images to extract features such as bone structure and personal color.

[0870] An "AI model" refers to an algorithm that uses machine learning or deep learning to analyze data and generate a specific diagnostic result.

[0871] "Comprehensive diagnosis results" refer to results that indicate the fashion items and brands that are best suited to a user, generated by integrating profile information, facial image analysis results, and the user's responses.

[0872] "Dialogue-style questions" refers to a format in which the system asks the user a series of questions and collects the user's answers.

[0873] "Smart glasses" refers to a glasses-type device that, when worn by a user, provides functions such as augmented reality and displays information in the user's field of vision.

[0874] A "virtual try-on experience" is an experience that allows users to get the feeling of trying on clothes in a virtual space without actually trying them on.

[0875] The system embodying this invention utilizes AI to help users find fashion and makeup items that suit them. The specific configuration of the system and its operation method are described below.

[0876] Server Operation

[0877] 1. Receiving profile information

[0878] The server receives the user's profile information (age, gender, fashion preferences, etc.) sent from the device and stores it in a database.

[0879] 2. Receiving and analyzing face images

[0880] After receiving the user's facial image from the device, the server applies image analysis algorithms to analyze the user's bone structure and personal color. The facial image analysis uses OpenCV and Keras models. The analysis results are temporarily stored.

[0881] 3. Interactive Question Generation and Answer Analysis

[0882] The server generates interactive questions and asks them to the user via the device, then uses an AI model to analyze the user's answers and dynamically generate the next questions until the required information is obtained.

[0883] 4. Generating comprehensive diagnostic results

[0884] The server integrates the received profile information, facial image analysis results, and the user's responses, and uses an AI model to generate a comprehensive diagnosis, which includes suitable fashion items, brands, and stores for the user.

[0885] 5. Presentation of results and recommendations

[0886] Based on the generated diagnostic results, the server generates suggestions for specific fashion items, brands, and stores, and transmits them to the terminal to present to the user.

[0887] Virtual try-on experience using smart glasses

[0888] 1. Capture images of your face and body

[0889] The smart glasses capture images of the user's face and body and send them to a server.

[0890] 2. Providing a virtual try-on experience

[0891] Based on the diagnostic results received from the server, the user is given a virtual fitting experience through the smart glasses, allowing them to try on clothes without actually going to a store.

[0892] Specific examples

[0893] Example 1: A woman in her 30s who likes casual fashion

[0894] 1. User: Puts on smart glasses, accesses the app, and enters age (30s), gender (female), and fashion preference (casual).

[0895] 2. Terminal: Sends input information to the server.

[0896] 3. User: Take a photo of their face with smart glasses and upload it.

[0897] 4. Server: Analyzes the facial image and determines that the user is a "spring type." It then generates a dialogue question, asking, "What color clothes do you like?"

[0898] 5. User: "I like pastel colors."

[0899] 6. Server: Based on the answers, identify pastel-colored casual fashion items and suitable brands and generate a diagnosis.

[0900] 7. Terminal: Displays diagnostic results and provides a virtual try-on experience.

[0901] 8. User: Virtually try on the item and decide to purchase from the suggested store.

[0902] Prompt Sentence Examples

[0903] Users enter their age, gender, fashion preferences, and upload a face image. They then answer the following interactive questions:

[0904] 1. What color clothes do you like?

[0905] 2. Where do you go on your days off?

[0906] 3. What style do you like?

[0907] Based on this information, we will display diagnostic results of recommended fashion items, brands, and shops.

[0908] This allows users to have a realistic try-on experience even remotely, and receive highly accurate fashion suggestions.

[0909] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0910] Step 1:

[0911] Enter profile information

[0912] The user puts on the smart glasses and accesses the application, entering their age, gender, and fashion preferences. This profile information is entered into the device and sent to the server.

[0913] Input: Age, Gender, Fashion Preferences

[0914] Output: Profile information sent to the server

[0915] Step 2:

[0916] Enter and send a facial image

[0917] Users use smart glasses to take a picture of their face and upload it to their device, which then sends the image to a server.

[0918] Input: User's face image

[0919] Output: Face image sent to the server

[0920] Step 3:

[0921] Facial image analysis

[0922] The server performs image analysis on the received facial image. This analysis uses OpenCV and Keras models to extract bone structure and personal color features. The analysis results are temporarily saved.

[0923] Input: Face image

[0924] Output: Bone structure and personal color analysis results

[0925] Step 4:

[0926] Interactive question generation

[0927] The server generates interactive questions based on the user's profile information and facial image analysis results, which are then sent to the device and displayed to the user.

[0928] Input: Profile information, facial image analysis results

[0929] Output: Question to the user

[0930] Step 5:

[0931] Receiving and analyzing user responses

[0932] Users answer questions displayed through their devices, and the answers are sent to a server that analyzes the user's answers, using a generative AI model to interpret the meaning of the answers and dynamically generate the next question.

[0933] Input: User's answer

[0934] Output: Parsed answer, next question

[0935] Step 6:

[0936] Generate comprehensive diagnostic results

[0937] The server integrates the received profile information, facial image analysis results, and the user's responses, and uses a generative AI model to generate a comprehensive diagnostic result, which includes suitable fashion items, brands, and stores for the user.

[0938] Input: Profile information, facial image analysis results, user responses

[0939] Output: Overall diagnostic results

[0940] Step 7:

[0941] Displaying diagnostic results

[0942] The server sends the generated diagnostic results to the device, which displays them to the user and prepares for the virtual try-on experience.

[0943] Input: Overall diagnostic results

[0944] Output: Display diagnostic results to the user

[0945] Step 8:

[0946] Providing a virtual try-on experience

[0947] The smart glasses provide users with a virtual try-on experience based on the diagnostic results received from the server, allowing them to try on clothes in a virtual space without actually going to a store.

[0948] Input: Diagnosis results, face and body images from smart glasses

[0949] Output: Virtual try-on experience

[0950] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0951] MODE FOR CARRYING OUT THE INVENTION

[0952] System program and processing description

[0953] The system of this invention utilizes AI and an emotion engine to help users find fashion and makeup items that suit them. The system operates in cooperation with a server, a terminal, and a user, each of which plays a specific role.

[0954] Server Operation

[0955] 1. Receiving profile information:

[0956] The server receives the user's profile information (age, gender, fashion preferences, etc.) sent from the device and stores it in a database.

[0957] 2. Receiving and analyzing face images:

[0958] The server receives the user's facial image from the device, applies an image analysis algorithm to analyze the user's bone structure and personal color, and temporarily stores the facial image analysis results.

[0959] 3. How the Emotion Engine works:

[0960] The server analyzes the user's emotions in real time based on the received facial images and dialogue content, and recognizes their emotional state, which is used in the next processing step.

[0961] 4. Interactive question generation and answer analysis:

[0962] The server generates interactive questions based on the emotion engine, taking into account the user's momentary emotional state, and asks the questions to the user via the device. It receives the user's answers, analyzes their content, and dynamically generates the next questions to ask. This process is repeated until all necessary information is obtained.

[0963] 5. Generating comprehensive diagnostic results:

[0964] The server integrates the received profile information, facial image analysis results, emotional state, and user responses, and uses an AI model to generate a comprehensive diagnostic result, which includes suitable fashion items, brands, and shops for the user.

[0965] 6. Presentation of results and recommendations:

[0966] Based on the generated diagnostic results, the server generates suggestions for specific fashion items, brands, and shops that take into account the emotional state, and transmits them to the terminal to present to the user.

[0967] Device behavior

[0968] 1. Provide profile information input interface:

[0969] The terminal provides an interface for the user to input profile information and transmits the input information to the server.

[0970] 2. Provide face image upload interface:

[0971] The terminal provides an interface for the user to upload a facial image and transmits the uploaded facial image to the server.

[0972] 3. Providing an interactive question and answer input interface:

[0973] The terminal displays the interactive questions sent from the server, provides an interface for the user to answer the questions, and sends the user's answers to the server.

[0974] 4. Provide diagnostic result display interface:

[0975] The terminal provides an interface for displaying the diagnosis results and suggestions sent from the server to the user.

[0976] User behavior

[0977] 1. Enter your profile information:

[0978] Through the device interface, users enter their profile information, including age, gender, and fashion preferences.

[0979] 2. Upload your face image:

[0980] Users upload their facial images using the device's interface.

[0981] 3. Answer the interactive questions:

[0982] The user answers questions sent from the server through the device. For example, to the question "What color clothes do you like?", the user answers "I like pastel colors."

[0983] 4. Review the diagnostic results and accept the recommendations:

[0984] Users can check the diagnostic results displayed on their device and browse suggested fashion items, brands, and shops.

[0985] Specific examples

[0986] Example 1: A woman in her 30s who likes casual fashion

[0987] 1. User: Accesses the app and enters age (30s), gender (female), and fashion preference (casual).

[0988] 2. Terminal: Sends input information to the server.

[0989] 3. User: Upload a photo of your face.

[0990] 4. Device: Sends a photo of your face to the server.

[0991] 5. Server: Performs facial image analysis and determines the person as a "spring type." The analysis results are temporarily saved.

[0992] 6. Server: Uses an emotion engine to recognize emotions from the user's facial images and answers, and understands their current emotional state.

[0993] 7. Server: Generates questions and asks the user, "What color clothes do you like?". Adjusts the questions depending on the user's emotional state.

[0994] 8. Terminal: displays the question and accepts the user's answer.

[0995] 9. User: "I like pastel colors."

[0996] 10. Terminal: Sends the answer to the server.

[0997] 11. Server: Analyzes the answer and generates the next question. Sends "What kind of clothes would you like to wear for what occasion?". Adjusts the next question based on the emotional state.

[0998] 12. Terminal: Show question.

[0999] 13. User: Enter "Relaxing holiday scene."

[1000] 14. Terminal: Sends the answer to the server, analyzes the answer, and repeats this process until all the necessary information is gathered.

[1001] 15. Server: Using an AI model based on facial image analysis results, answers to interactive questions, and emotional state, the server generates diagnostic results suitable for casual fashion, including information on "pastel-colored items" and "suitable brands."

[1002] 16. Server: Dynamically generates diagnostic results and specific recommendations.

[1003] 17. Terminal: Displays the diagnostic results and suggestions to the user.

[1004] 18. User: Based on the presented results, select a pastel-colored casual fashion item and purchase it from the suggested shop.

[1005] In this way, by combining emotion engines, it is possible to provide highly accurate personalized fashion advice according to the user's emotional state.

[1006] The processing flow will be explained below.

[1007] Program processing steps

[1008] Step 1: Enter your user information

[1009] User: Visits an app or website and enters profile information such as age, gender, and fashion preferences.

[1010] Terminal: Receives profile information entered by the user and sends it to the server.

[1011] Server: Stores the received profile information in a database and prepares for the next step.

[1012] Step 2: Upload and analyze face images

[1013] Users: Upload a photo of themselves to an app or website.

[1014] Terminal: Sends the uploaded face photo to the server.

[1015] Server: Analyzes bone structure and personal color using facial image analysis algorithms. Saves the analysis results temporarily and uses them in the next step.

[1016] Step 3: Emotion analysis using the emotion engine

[1017] Server: Analyzes the user's emotions in real time using an emotion engine based on facial images and dialogue content. Temporarily stores the analysis results.

[1018] Step 4: Interactive question generation and answer analysis

[1019] Server: Generates conversational questions that take into account the user's emotional state and sends them to the device. For example, set questions such as "What color clothes do you like?" or "What kind of clothes do you like to wear for what occasions?"

[1020] Terminal: Presents an interface for displaying interactive questions to the user and entering answers.

[1021] User: Enters an answer to a question.

[1022] Terminal: Sends the entered answer to the server.

[1023] Server: Analyzes the received answers and dynamically generates the next question to ask. This process is repeated until all necessary questions have been asked.

[1024] Step 5: Generate comprehensive diagnostic results

[1025] Server: Integrates the results of facial image analysis, answers collected through dialogue questions, and emotional state, and uses AI models to generate comprehensive diagnostic results. This identifies the user's personal color, clothing styles and designs that suit them, popular brands, and suitable accessories and makeup items.

[1026] Server: Stores the diagnostic results in a database and prepares the results for display.

[1027] Step 6: Present your results and make recommendations

[1028] Server: Based on the diagnosis results, it generates recommendations for specific fashion items, brands, and shops and sends them to the device. The recommendations may be updated in real time.

[1029] Terminal: Provides an interface for displaying the diagnostic results and suggestions sent from the server to the user.

[1030] User: Based on the results presented, users can choose fashion and makeup items that suit them best.

[1031] Specific examples

[1032] Example: A woman in her 30s who likes casual fashion

[1033] 1. Step 1:

[1034] User: Accesses the app and enters age (30s), gender (female), and fashion preference (casual).

[1035] Terminal: Sends input information to the server.

[1036] Server: Saves the input information in a database.

[1037] 2. Step 2:

[1038] User: Upload a photo of themselves to the app.

[1039] Device: Sends a photo of your face to the server.

[1040] Server: Performs facial image analysis and determines "spring type." Temporarily saves the analysis results.

[1041] 3. Step 3:

[1042] Server: Uses an emotion engine to recognize emotions from the user's facial image and responses, and understands their current emotional state. For example, it determines whether the user is in a "peace of mind" state.

[1043] 4. Step 4:

[1044] Server: Based on the emotional state, generate a question like "What color clothes do you like?" and send it to the user. The question content is adjusted depending on the emotional state.

[1045] Terminal: Provides an interface for displaying questions and accepting user answers.

[1046] User: "I like pastel colors."

[1047] Terminal: Sends the answer to the server.

[1048] Server: Analyzes the answer and generates the next question: "What kind of occasions do you want to wear your clothes for?" The next question is adjusted taking into account the emotional state.

[1049] Terminal: Display the question.

[1050] User: Enter "Relaxing holiday scene."

[1051] Device: Sends the answer to the server and repeats this process until it has gathered the required information.

[1052] 5. Step 5:

[1053] Server: Using an AI model based on facial image analysis, responses to interactive questions, and emotional state, the server generates a comprehensive diagnosis suitable for casual fashion, including information on "pastel-colored items" and "suitable brands."

[1054] Server: Stores the diagnostic results in a database.

[1055] 6. Step 6:

[1056] Server: Generates diagnostic results and specific fashion item, brand, and shop recommendations based on the results. Suggestions may be updated in real time.

[1057] Terminal: Provides an interface for displaying diagnostic results and recommendations to the user.

[1058] User: Based on the presented results, select a pastel-colored casual fashion item and purchase it from the suggested shop.

[1059] In this way, users can receive highly accurate and realistic fashion advice that takes their emotions into consideration.

[1060] Example 2

[1061] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1062] Conventional fashion advice systems often make suggestions based solely on the user's profile information or image analysis results, and are unable to consider the user's emotional state or real-time changes in preferences. This makes it difficult to provide personalized, highly accurate fashion advice, resulting in reduced user satisfaction. The present invention aims to solve these problems and provide personalized fashion advice that takes the user's emotional state into account.

[1063] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1064] In this invention, the server includes a means for receiving input information from a user, a means for receiving a facial image of the user and analyzing facial features and color characteristics through image processing, and a means for analyzing the user's emotions in real time and dynamically generating questions based on the analysis results. This makes it possible to integrate the user's profile information, facial image analysis results, emotion analysis results, and user answers, use a generative AI model to generate highly accurate comprehensive diagnostic results, and suggest clothing items, brands, and sales locations that are suitable for the user.

[1065] "Input Information" means the personal information and profile data that a User provides to the System.

[1066] A "face image" refers to a photograph or image data of a user's face.

[1067] "Image processing" refers to the technology of analyzing image data and extracting specific features or attributes.

[1068] "Facial features" refers to the shape, bone structure, and feature points of the user's face.

[1069] "Color characteristics" refers to a personal color classification based on the user's face and skin color.

[1070] "Emotion analysis" is a technology that analyzes a user's emotional state in real time from facial images and dialogue content.

[1071] A "generative AI model" is an artificial intelligence model that uses machine learning and natural language processing to analyze data and generate interactive questions and diagnostic results.

[1072] "Comprehensive diagnosis results" are diagnostic results generated by the AI ​​model based on the user's profile information, facial image analysis results, emotion analysis results, and the user's responses.

[1073] "Clothing items" refers to clothes, accessories, etc. suggested to users.

[1074] "Trademark" means the name or logo of a particular brand or manufacturer, which is associated with the proposed fashion item.

[1075] "Point of Sale" refers to a store or online shop where the products suggested to the user can be purchased.

[1076] This invention is a system that utilizes AI and an emotion engine to help users find fashion and makeup items that suit them. The system works in cooperation with a server, a device, and a user, each of which plays a specific role.

[1077] Server Operation

[1078] 1. Receiving profile information

[1079] The server receives the user's input information (age, gender, fashion preferences, etc.) sent from the device and stores it in a database. Specifically, it uses a MySQL database.

[1080] 2. Receiving and analyzing face images

[1081] After receiving the user's facial image from the device, the server processes the image using OpenCV and Dlib. The facial image is analyzed to analyze the user's skeletal and color characteristics, and the analysis results are temporarily stored in Redis.

[1082] 3. Operation of the Emotion Engine

[1083] The server uses Microsoft Azure's emotion analysis API to analyze the user's emotions in real time from facial images and dialogue content, recognizing their emotional state. The analysis results are used in the next step of question generation.

[1084] 4. Question Generation and Answer Analysis

[1085] The server generates questions that take into account the user's temporary emotional state based on the results of sentiment analysis. The generated questions are sent to the user via the device and the user's answers are received. The received answers are then analyzed and the next question is dynamically generated. The NLP technology used is SpaCy and the BERT model.

[1086] 5. Generating comprehensive diagnostic results

[1087] The server integrates the received input information, facial image analysis results, emotional state, and user responses, and uses a generative AI model (e.g., GPT-4) to generate a comprehensive diagnosis result, which includes clothing items, brands, and sales locations suitable for the user.

[1088] 6. Presentation of results and recommendations

[1089] Based on the generated diagnostic results, the server generates specific suggestions that take into account the emotional state, and transmits them to the terminal to present to the user.

[1090] Device behavior

[1091] 1. Provides a profile information input interface

[1092] The terminal provides an interface for users to input information and sends the input information to the server, specifically using JavaScript and ReactJS.

[1093] 2. Providing a face image upload interface

[1094] The terminal provides an interface for the user to upload a facial image and transmits the uploaded facial image to the server.

[1095] 3. Providing a question and answer input interface

[1096] The terminal provides an interface for displaying the questions sent from the server and accepting the user's answers, and then sends the user's answers to the server.

[1097] 4. Providing an interface for displaying diagnostic results

[1098] The terminal provides an interface for displaying the diagnosis results and suggestions sent from the server to the user.

[1099] User behavior

[1100] 1. Enter your profile information

[1101] Through the device's interface, users enter their personal information, including age, gender, and fashion preferences.

[1102] 2. Upload a face image

[1103] Users upload their facial images using the device's interface.

[1104] 3. Answering questions

[1105] The user answers questions sent from the server through the device. For example, to the question "What color clothes do you like?", the user answers "I like pastel colors."

[1106] 4. Review the diagnostic results and accept the recommendations

[1107] The user checks the diagnostic results displayed on the device and browses suggested clothing items, brands, and sales locations.

[1108] Specific examples

[1109] Example 1: A woman in her 30s who likes casual fashion

[1110] 1. User: Accesses the app and enters age (30s), gender (female), and fashion preference (casual).

[1111] 2. Terminal: Sends input information to the server.

[1112] 3. Server: Stores profile information in a MySQL database.

[1113] 4. User: Upload a photo of your face.

[1114] 5. Device: Sends a photo of your face to the server.

[1115] 6. Server: Performs facial image analysis and determines "spring type." Temporarily stores the analysis results in Redis.

[1116] 7. Server: Uses an emotion engine to recognize emotions from the user's facial images and answers, and understands their current emotional state.

[1117] 8. Server: Generates questions and asks the user, "What color clothes do you like?". Adjusts the questions depending on the user's emotional state.

[1118] 9. Terminal: displays the question and accepts the user's answer.

[1119] 10. User: "I like pastel colors."

[1120] 11. Terminal: Sends the answer to the server.

[1121] 12. Server: Analyzes the answer and generates the next question. Sends "What kind of clothes would you like to wear for what occasion?". Adjusts the next question based on the emotional state.

[1122] 13. Terminal: Display the question.

[1123] 14. User: Enter "Relaxing holiday scene."

[1124] 15. Terminal: Sends the answer to the server, analyzes the answer, and repeats this process until all the necessary information is gathered.

[1125] 16. Server: Using a generative AI model based on facial image analysis results, answers to interactive questions, and emotional state, the server generates diagnostic results suitable for casual fashion, including information on "pastel-colored items" and "suitable brands."

[1126] 17. Server: Dynamically generates diagnostic results and specific recommendations.

[1127] 18. Terminal: Display the diagnostic results and suggestions to the user.

[1128] 19. User: Based on the presented results, select a pastel-colored casual fashion item and purchase it at the suggested retail location.

[1129] Prompt Sentence Examples

[1130] "I'm a woman in my 30s who likes casual fashion. My face image indicates I'm a spring type. What fashion items and brands would you suggest?"

[1131] In this way, it is possible to provide personalized fashion advice according to the user's emotional state.

[1132] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1133] Step 1:

[1134] Enter profile information

[1135] The user enters their own information (age, gender, fashion preferences, etc.) through the device interface. The device validates the entered information using JavaScript and sends it to the server. The server receives the input information and saves it in a MySQL database using INSERT statements. The input is the user's profile information, and the output is the profile information saved in the database.

[1136] Step 2:

[1137] Upload a face image

[1138] A user uploads their facial image using the device interface. The device encodes the uploaded facial image into Base64 format and sends it to the server. The server receives the facial image and temporarily stores it in a file system or database. The input is the user's facial image, and the output is the facial image stored on the server.

[1139] Step 3:

[1140] Facial image analysis

[1141] The server analyzes the stored facial images using OpenCV and Dlib. First, it uses a face detection algorithm to extract facial feature points, then analyzes the bone structure and color characteristics. The analysis results are temporarily stored in Redis. The input is the user's facial image, and the output is the analysis results including bone structure and color characteristics.

[1142] Step 4:

[1143] Emotion analysis

[1144] The server uses Microsoft Azure's emotion analysis API to analyze the user's emotions in real time from facial images and dialogue content. It makes API calls, obtains the analysis results, and stores them in a database. The input is the user's facial image and dialogue content, and the output is the analysis results that indicate the user's emotional state.

[1145] Step 5:

[1146] Question generation and answer analysis

[1147] The server takes into account the user's emotional state based on the sentiment analysis results and uses a generative AI model to dynamically generate the next question. It then sends the generated question to the user via their device and receives the user's answer. It analyzes the answer and, if necessary, generates the next question and repeats the process. The input is the sentiment analysis result and the user's answer, and the output is the dynamically generated next question.

[1148] Step 6:

[1149] Generate comprehensive diagnostic results

[1150] The server integrates the profile information, facial image analysis results, emotion analysis results, and the user's responses, and uses a generative AI model to generate a comprehensive diagnosis. Specifically, a prompt is entered to query the AI ​​model and obtain the results. The diagnosis results include clothing items, trademarks, and sales locations suitable for the user. The input is the integrated user information, and the output is the comprehensive diagnosis results.

[1151] Step 7:

[1152] Presentation of results and recommendations

[1153] The server sends the generated diagnostic results to the terminal. The terminal provides an interface to display the diagnostic results and suggestions to the user. The user can review the presented results and select items that suit their preferences. The input is the overall diagnostic results, and the output is the suggestions displayed to the user.

[1154] (Application example 2)

[1155] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1156] Conventional fashion and makeup item recommendation systems have issues with insufficient suggestions based on user profile information or facial image analysis, and are unable to provide personalized advice that takes into account the user's emotional state. Furthermore, they lack the ability to dynamically generate questions that respond to the user's preferences and emotions, or to adjust the recommendations in real time, making it difficult to recommend optimal products to users.

[1157] The identification processing by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for receiving profile information from a user, means for receiving a facial image of the user and analyzing the user's bone structure and personal color through image analysis, means for interactively asking the user questions and analyzing the user's responses, means for integrating the received profile information, facial image analysis results, and the user's responses and generating a comprehensive diagnostic result using an AI model, means for proposing products suitable for the user based on the diagnostic result, means for analyzing the user's emotional state in real time and dynamically adjusting the content of the interactive questions based on the emotional state, and means for inputting prompt sentences into a generative AI model generated based on the emotional state and profile information to make personalized suggestions. This enables optimal product suggestions in real time while taking the user's emotional state into consideration.

[1158] "Profile Information" is information about a user's personal information, such as their age, gender, and fashion preferences.

[1159] "Bone structure" refers to features that represent the structure of a user's face and body.

[1160] "Personal Color" refers to the range of colors that best complement a user's natural skin tone and hair color.

[1161] "Dialogue" is a format in which the user and the system collect information by exchanging questions and answers.

[1162] An "AI model" is a mathematical model that uses machine learning and artificial intelligence techniques to analyze data and make recommendations and predictions.

[1163] The "comprehensive diagnosis result" is the result of optimal suggestions for the user, generated based on the received profile information, facial image analysis results, and user responses.

[1164] "Emotional state" indicates the user's current emotional state and is estimated from facial expressions and dialogue content.

[1165] A "generative AI model" is an AI model that generates personalized suggestions based on a user's profile information and emotional state using specific prompt sentences as input.

[1166] A "prompt sentence" is text that is input into a generative AI model and serves as the basis for generating specific suggestions for the user.

[1167] "Dynamic adjustment" means making changes and adaptations instantly in response to the user's real-time reactions and conditions.

[1168] This invention relates to a system that enables users to find fashion items and makeup items that suit them, and is mainly comprised of a server, a terminal, and a user, each of which plays a specific role and operates in cooperation with one another.

[1169] System configuration

[1170] Server Operation

[1171] 1. The server receives the user's profile information (age, gender, fashion preferences, etc.) sent from the device and stores it in a database.

[1172] 2. After receiving the user's facial image from the device, the server applies image analysis algorithms to analyze the facial structure and personal color. This image analysis uses image processing libraries such as OpenCV.

[1173] 3. The server uses the Emotion Engine to analyze the user's emotions in real time based on the received facial images and dialogue content, and recognizes their emotional state.

[1174] 4. The server generates dialogue-style questions based on the emotion engine, taking into account the user's temporary emotional state, and asks the questions to the user via the terminal. This can be done using a framework such as Flask or Django.

[1175] 5. The server integrates the received profile information, facial image analysis results, emotional state, and the user's answers, and generates a comprehensive diagnosis result using an AI model called AIFashionModel, which includes fashion items, brands, and shops suitable for the user.

[1176] 6. The server generates suggestions for specific fashion items, brands, and shops that take into account the emotional state, and sends them to the terminal to present to the user.

[1177] Device behavior

[1178] 1. The terminal provides an interface for the user to input profile information and transmits the input information to the server.

[1179] 2. The terminal provides an interface for the user to upload a facial image and transmits the uploaded facial image to the server.

[1180] 3. The terminal displays the interactive questions sent from the server, provides an interface for the user to answer the questions, and sends the user's answers to the server.

[1181] 4. The terminal provides an interface for displaying the diagnostic results and suggestions sent from the server to the user.

[1182] User behavior

[1183] 1. Through the device interface, users enter their profile information, including age, gender, and fashion preferences.

[1184] 2. The user uploads an image of their face using the device interface.

[1185] 3. The user answers questions sent from the server through their device. For example, to the question "What color clothes do you like?", the user answers "I like pastel colors."

[1186] 4. The user checks the diagnostic results displayed on the device and browses suggested fashion items, brands, and shops.

[1187] Specific examples

[1188] For example, for a female user in her 30s who likes casual fashion,

[1189] 1. The user enters their age (30s), gender (female), and fashion preference (casual).

[1190] 2. Upload a photo of your face and your face will be analyzed to determine your "spring type."

[1191] 3. The server uses an emotion engine to recognize the user's "current emotional state" from the facial image and generates interactive questions.

[1192] 4. For example, ask the user, "What color clothes do you like?"

[1193] 5. The user answers, "I like pastel colors," and based on this, the next question is dynamically generated, such as, "What kind of clothes do you like to wear for what occasions?"

[1194] 6. This process is repeated until all the necessary information is gathered, and finally, personalized fashion items are suggested.

[1195] Prompt Sentence Examples

[1196] If the profile information is "30s, female, casual fashion" and the emotional state is "happy," the server will input the following prompt sentence to the AI ​​model:

[1197] "A woman in her 30s likes casual fashion and spring-type pastel colors. She is currently in a happy emotional state. Please suggest some fashion items that would be perfect for relaxing on the weekend."

[1198] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1199] Step 1:

[1200] Users enter profile information through the device interface, such as age, gender, and fashion preferences. This information becomes input data, which the device receives and transmits to the server.

[1201] Step 2:

[1202] The server stores the user profile information received from the device in a database, which will be referenced in subsequent processing.

[1203] Step 3:

[1204] The user takes a picture of their face using the device's camera and uploads it to the server. The face image becomes input data, and the device receives this face image data and sends it to the server.

[1205] Step 4:

[1206] The server analyzes the received facial images using an image processing library such as OpenCV. This analysis identifies the facial bone structure and personal color, which are then temporarily saved as intermediate data.

[1207] Step 5:

[1208] The server uses EmotionEngine to analyze the user's emotional state in real time based on the received facial images and dialogue content. The emotional state is output as an analysis result and used in the next step.

[1209] Step 6:

[1210] Based on the analyzed emotional state, the server dynamically generates interactive questions, which are tailored to the user's current emotions and sent to the device as question data.

[1211] Step 7:

[1212] The terminal presents the user with interactive questions sent from the server, and the user answers them, and the terminal receives the answer data and sends it to the server.

[1213] Step 8:

[1214] The server analyzes the user's answers and dynamically generates subsequent questions to obtain further required information. This process is repeated until the required information is gathered.

[1215] Step 9:

[1216] The server integrates the received profile information, facial image analysis results, emotional state, and user responses, and generates a comprehensive diagnosis result using the AIFashionModel, which includes a list of products suitable for the user, and generates this data as recommendation data.

[1217] Step 10:

[1218] The server takes into account the emotional state and diagnosis results and inputs the optimal suggestion into the generative AI model as a prompt sentence, which contains specific fashion item suggestions for the user.

[1219] Step 11:

[1220] The generated suggestions are sent to the device and presented to the user. The user can use these suggestions as a reference when selecting products. For example, a prompt might read, "A woman in her 30s likes casual fashion and spring-type pastel colors. Her current emotional state is happy. Please suggest the best fashion items for a relaxing holiday."

[1221] This makes it possible to provide personalized fashion advice and suggestions that take into account emotional state in real time.

[1222] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1223] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1224] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[1225] [Third embodiment]

[1226] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[1227] 5, the data processing system 310 includes the data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[1228] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1229] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[1230] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1231] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1232] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1233] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1234] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1235] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1236] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1237] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[1238] MODE FOR CARRYING OUT THE INVENTION

[1239] System program and processing description

[1240] The system of this invention utilizes AI to help users find fashion and makeup items that suit them. The system operates in cooperation with a server, a terminal, and a user, each of which plays a specific role.

[1241] Server Operation

[1242] 1. Receiving profile information:

[1243] The server receives the user's profile information (age, gender, fashion preferences, etc.) sent from the device and stores it in a database.

[1244] 2. Receiving and analyzing face images:

[1245] The server receives the user's facial image from the device, applies an image analysis algorithm to analyze the user's bone structure and personal color, and temporarily stores the facial image analysis results.

[1246] 3. Interactive question generation and answer analysis:

[1247] The server generates interactive questions and asks them to the user via the terminal, receives the user's answers, analyzes the answers, and dynamically generates the next questions to ask. This process is repeated until all necessary information is obtained.

[1248] 4. Generating comprehensive diagnostic results:

[1249] The server integrates the received profile information, facial image analysis results, and user responses, and uses an AI model to generate a comprehensive diagnosis, which includes suitable fashion items, brands, and shops for the user.

[1250] 5. Presentation of results and recommendations:

[1251] Based on the generated diagnostic results, the server generates suggestions for specific fashion items, brands, and shops, and transmits them to the terminal to present to the user.

[1252] Device behavior

[1253] 1. Provide profile information input interface:

[1254] The terminal provides an interface for the user to input profile information and transmits the input information to the server.

[1255] 2. Provide face image upload interface:

[1256] The terminal provides an interface for the user to upload a facial image and transmits the uploaded facial image to the server.

[1257] 3. Providing an interactive question and answer input interface:

[1258] The terminal displays the questions sent from the server, provides an interface for the user to answer the questions, and sends the user's answers to the server.

[1259] 4. Provide diagnostic result display interface:

[1260] The terminal provides an interface for displaying the diagnosis results and suggestions sent from the server to the user.

[1261] User behavior

[1262] 1. Enter your profile information:

[1263] Through the device interface, users enter their profile information, including age, gender, and fashion preferences.

[1264] 2. Upload your face image:

[1265] Users upload their facial images using the device's interface.

[1266] 3. Answer the interactive questions:

[1267] The user answers questions sent from the server through the device. For example, to the question "What color clothes do you like?", the user answers "I like pastel colors."

[1268] 4. Review the diagnostic results and accept the recommendations:

[1269] Users can check the diagnostic results displayed on their device and browse suggested fashion items, brands, and shops.

[1270] Specific examples

[1271] Example 1: A woman in her 30s who likes casual fashion

[1272] 1. User: Accesses the app and enters age (30s), gender (female), and fashion preference (casual).

[1273] 2. Terminal: Sends input information to the server.

[1274] 3. User: Upload a photo of your face.

[1275] 4. Device: Send a photo of your face to the server.

[1276] 5. Server: Performs facial image analysis and determines the user is a "spring type." Generates a dialogue question, asking, "What color clothes do you like?"

[1277] 6. User: "I like pastel colors."

[1278] 7. Terminal: Sends the answer to the server.

[1279] 8. Server: Based on all the information, identify pastel-colored casual fashion items and suitable brands and generate a diagnosis.

[1280] 9. Terminal: Display the diagnostic results.

[1281] 10. User: Select an item based on the diagnosis results and purchase it from the suggested shop.

[1282] Example 2: A man in his 40s who likes formal fashion

[1283] 1. User: Accesses the app and enters age (40s), gender (male), and fashion preference (formal).

[1284] 2. Terminal: Sends input information to the server.

[1285] 3. User: Upload a photo of your face.

[1286] 4. Device: Send a photo of your face to the server.

[1287] 5. Server: Performs facial image analysis and determines the person is a "winter type." Generates a dialogue question, asking, "What color do you prefer for formal occasions?"

[1288] 6. User: "Dark blue or black."

[1289] 7. Terminal: Sends the answer to the server.

[1290] 8. Server: Based on all the information, it identifies formal fashion items and popular brands and generates diagnostic results.

[1291] 9. Terminal: Display the diagnostic results.

[1292] 10. User: Select an item based on the diagnosis results and purchase it from the suggested shop.

[1293] In this way, users can easily find the fashion style that suits them through highly accurate diagnosis. The system handles everything from entering profile information, analyzing facial images, asking interactive questions, and generating and proposing comprehensive diagnosis results.

[1294] The processing flow will be explained below.

[1295] Program processing steps

[1296] Step 1: Enter your user information

[1297] User: Visits an app or website and enters profile information such as age, gender, and fashion preferences.

[1298] Terminal: Receives profile information entered by the user and sends it to the server.

[1299] Server: Stores the received profile information in a database and prepares for the next step.

[1300] Step 2: Upload and analyze face images

[1301] Users: Upload a photo of themselves to an app or website.

[1302] Terminal: Sends the uploaded face photo to the server.

[1303] Server: Analyzes bone structure and personal color using facial image analysis algorithms. Saves the analysis results temporarily and uses them in the next step.

[1304] Step 3: Interactive question generation and answer analysis

[1305] Server: Generates interactive questions and sends them to the device. Initial questions include "What color clothes do you like?" and "What kind of clothes do you like to wear for what occasions?"

[1306] Terminal: Provides an interface for displaying interactive questions to the user and for entering answers.

[1307] User: Enters an answer to a question.

[1308] Terminal: Sends the entered answer to the server.

[1309] Server: Analyzes the received answers and dynamically generates the next question to ask. This process is repeated until all necessary questions have been asked.

[1310] Step 4: Generate comprehensive diagnostic results

[1311] Server: The server combines the results of facial image analysis with the user's responses to interactive questions and uses an AI model to generate a comprehensive diagnosis, specifically identifying the user's personal color, clothing styles and designs that suit them, popular brands, and suitable accessories and makeup items.

[1312] Server: Stores the diagnostic results in a database and prepares the results for display.

[1313] Step 5: Present the results and make recommendations

[1314] Server: Generates the diagnosis results and specific fashion item, brand, and shop recommendations based on them. The recommendations may be updated in real time, so they must be generated dynamically.

[1315] Terminal: Provides an interface for presenting diagnostic results to the user.

[1316] User: Based on the results presented, users can choose fashion and makeup items that suit them best.

[1317] Specific examples

[1318] Example: A woman in her 30s who likes casual fashion

[1319] 1. Step 1:

[1320] User: Accesses the app and enters age (30s), gender (female), and fashion preference (casual).

[1321] Terminal: Sends input information to the server.

[1322] Server: Saves the input information in a database.

[1323] 2. Step 2:

[1324] User: Upload a photo of themselves to the app.

[1325] Device: Sends a photo of your face to the server.

[1326] Server: Performs facial image analysis and determines "spring type." Temporarily saves the analysis results.

[1327] 3. Step 3:

[1328] Server: Generates a dialogue question and asks the user, "What color clothes do you like?"

[1329] Terminal: Displays questions and accepts user answers.

[1330] User: "I like pastel colors."

[1331] Terminal: Sends the answer to the server.

[1332] Server: Analyzes the answer and generates the next question: "What kind of clothes would you like to wear for what occasion?"

[1333] Terminal: Show question.

[1334] User: Enter "Relaxing holiday scene."

[1335] Device: Sends the answer to the server, analyzes the answer, and repeats this process until it has gathered all the information it needs.

[1336] 4. Step 4:

[1337] Server: Using an AI model based on the results of facial image analysis and answers to interactive questions, the server generates diagnostic results suitable for casual fashion, including information on "pastel-colored items" and "suitable brands."

[1338] Server: Stores the diagnostic results in a database.

[1339] 5. Step 5:

[1340] Server: Dynamically generates diagnostic results and recommendations for specific fashion items, brands, and shops based on those results.

[1341] On the device: Display diagnostic results and suggestions to the user.

[1342] User: Based on the presented results, select a pastel-colored casual fashion item and purchase it from the suggested shop.

[1343] In this way, users can receive highly accurate and realistic fashion advice throughout the process.

[1344] Example 1

[1345] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1346] Users often spend a lot of time and effort finding the perfect fashion and makeup items for themselves, and it can be particularly difficult to choose items that take into account facial features and personal color. Current systems are unable to accurately analyze these points and make real-time suggestions that match the user's preferences and characteristics. This leaves users with the challenge of being unable to quickly and easily find the perfect fashion and makeup items based on their preferences and characteristics.

[1347] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1348] In this invention, the server includes means for receiving profile information from a user, means for receiving a facial image of the user and analyzing bone structure and personal color through image analysis, means for interactively asking the user questions and analyzing the user's responses, means for integrating the received profile information, facial image analysis results, and user responses and generating a comprehensive diagnostic result using an artificial intelligence model, means for suggesting clothing items, brands, and retailers suitable for the user based on the diagnostic result, communication means for sending initial information to the server in JSON format, means for sending a facial image via an HTTP request, means for analyzing the facial image using an image analysis library, means for dynamically generating interactive questions using the profile information and image analysis results, and means for generating and presenting a comprehensive diagnostic result, thereby enabling users to quickly and easily find fashion and makeup items based on their characteristics and preferences.

[1349] "Profile information" refers to information about a user's personal characteristics, such as the user's age, gender, and fashion preferences.

[1350] A "face image" is an image of the user's face, and facial features and personal color are analyzed based on this image.

[1351] "Image analysis" is a process performed on a received facial image to identify the user's bone structure and personal color.

[1352] "Dialogue format" refers to a format in which questions are asked to the user sequentially, and the next question is dynamically generated based on the answers.

[1353] An "artificial intelligence model" is a model that uses technologies such as machine learning and deep learning, and generates comprehensive diagnostic results based on the received data.

[1354] The "comprehensive diagnosis results" are generated by integrating profile information, facial image analysis results, and user responses, and include suggestions for fashion items, trademarks, and retailers that are suitable for the user.

[1355] "Communication means" refers to the technical means for transmitting initial information and analysis data to the server in JSON format or via HTTP requests.

[1356] The "JSON format" is a format that structures and expresses data in text format, and is primarily used for sending and receiving data.

[1357] An "HTTP request" is a protocol that allows a client to request a server to send data or obtain information.

[1358] An "image analysis library" is a software library, such as OpenCV or Dlib, that is used to extract and analyze features from facial images.

[1359] "Dynamic generation" means adaptively generating the next question or suggestion on the spot based on the user's answers.

[1360] "Diagnosis results" are recommendations for fashion items and services suitable for the user, generated based on analysis of the received data and artificial intelligence models.

[1361] MODE FOR CARRYING OUT THE INVENTION

[1362] This invention provides a system in which a server, a terminal, and a user work together to help users find the fashion and makeup items that are best suited to them. The overall configuration of the system and the operation of each component are described in detail below.

[1363] Server Operation

[1364] 1. Receiving and storing profile information:

[1365] The server receives the user's profile information (age, gender, fashion preferences, etc.) sent from the device and stores it in a database (e.g., MySQL or PostgreSQL), making it possible to manage each user's individual information.

[1366] 2. Receiving and analyzing face images:

[1367] After receiving the user's facial image from the device, the server analyzes the user's bone structure and personal color using image analysis libraries such as OpenCV and Dlib. The analysis results are temporarily stored in a database.

[1368] 3. Interactive question generation and answer analysis:

[1369] The server uses an NLP library (e.g., NLTK or SpaCy) to generate interactive questions and pose them to the user through the terminal. After receiving the user's answers, it analyzes their content and dynamically generates the next questions.

[1370] 4. Generating comprehensive diagnostic results:

[1371] The server integrates the profile information, facial image analysis results, and user responses, and generates a comprehensive diagnostic result using an AI model (e.g., TensorFlow or PyTorch), which can identify suitable fashion and makeup items for the user.

[1372] 5. Presentation of results and recommendations:

[1373] The server generates specific suggestions based on the generated diagnostic results and sends them to the terminal for presentation to the user, including clothing items, brands, and retailers suitable for the user.

[1374] Device behavior

[1375] 1. Provide profile information input interface:

[1376] The terminal provides an interface for the user to input profile information and transmits the input information to the server.

[1377] 2. Provide face image upload interface:

[1378] The terminal provides an interface for the user to upload a facial image and transmits the uploaded facial image to the server.

[1379] 3. Providing an interactive question and answer input interface:

[1380] The terminal displays the questions sent from the server, provides an interface for the user to answer the questions, and sends the user's answers to the server.

[1381] 4. Provide diagnostic result display interface:

[1382] The terminal provides an interface for displaying the diagnosis results and suggestions sent from the server to the user.

[1383] User behavior

[1384] 1. Enter your profile information:

[1385] Users enter their profile information through the device interface, including their age, gender and fashion preferences.

[1386] 2. Upload your face image:

[1387] Users upload their facial images using the device's interface.

[1388] 3. Answer the interactive questions:

[1389] The user answers questions sent from the server through the device. For example, to the question "What color clothes do you like?", the user answers "I like pastel colors."

[1390] 4. Review the diagnostic results and accept the recommendations:

[1391] Users can check the diagnosis results displayed on their device and use the suggested fashion and makeup items as a reference.

[1392] Specific use cases

[1393] Example 1: A woman in her 30s who likes casual fashion

[1394] 1. User: Accesses the app and enters age (30s), gender (female), and fashion preference (casual).

[1395] 2. Terminal: Sends input information to the server.

[1396] 3. User: Upload a photo of yourself.

[1397] 4. Device: Sends a photo of your face to the server.

[1398] 5. Server: Performs facial image analysis and determines the user is a "spring type." Generates a dialogue question, asking, "What color clothes do you like?"

[1399] 6. User: "I like pastel colors."

[1400] 7. Terminal: Sends the answer to the server.

[1401] 8. Server: Based on all the information, it identifies pastel-colored casual fashion items and suitable trademarks and generates a diagnosis result.

[1402] 9. Terminal: Display the diagnostic results.

[1403] 10. User: Select an item based on the diagnosis results and purchase it from the suggested retailer.

[1404] Example 2: A man in his 40s who likes formal fashion

[1405] 1. User: Accesses the app and enters age (40s), gender (male), and fashion preference (formal).

[1406] 2. Terminal: Sends input information to the server.

[1407] 3. User: Upload a photo of yourself.

[1408] 4. Device: Sends a photo of your face to the server.

[1409] 5. Server: Performs facial image analysis and determines the person is a "winter type." Generates a dialogue question, asking, "What color do you prefer for formal occasions?"

[1410] 6. User: "I like dark blue and black."

[1411] 7. Terminal: Sends the answer to the server.

[1412] 8. Server: Based on all the information, it identifies formal fashion items and popular brands and generates diagnostic results.

[1413] 9. Terminal: Display the diagnostic results.

[1414] 10. User: Select an item based on the diagnosis results and purchase it from the suggested retailer.

[1415] In this way, users can quickly and easily find the fashion and makeup items that best suit them through the system.

[1416] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1417] Step 1: Enter your profile information

[1418] User Action: The user enters their profile information (age, gender, fashion preferences) into the interface provided by the device.

[1419] Input: User profile information (e.g., "30s," "Female," "Casual")

[1420] Output: Sending profile information from device to server

[1421] Specific action: Enter data into a form in a web application and press the "Submit" button.

[1422] Step 2: Submit your profile information

[1423] Device behavior: The device sends the entered profile information to the server in JSON format.

[1424] Input: Profile information entered by the user

[1425] Output: Profile information received by the server

[1426] What it does: Sends profile information to the / api / profile endpoint using an HTTP POST request.

[1427] Step 3: Upload your face image

[1428] User action: The user selects and uploads a face image using the device interface.

[1429] Input: A face image selected by the user

[1430] Output: Sending face image from device to server

[1431] Specific operation: Select an image file and press the "Upload" button.

[1432] Step 4: Send a face image

[1433] Device operation: The device sends the uploaded facial image to the server.

[1434] Input: Face image file uploaded by the user

[1435] Output: Face image received by the server

[1436] What it does: Sends an image file to the / api / upload endpoint using an HTTP POST request.

[1437] Step 5: Facial image analysis

[1438] Server operation: The server analyzes the received facial image using an image analysis library such as OpenCV or Dlib to identify bone structure and personal color.

[1439] Input: Face image sent to the server

[1440] Output: Analysis results of bone structure and personal color

[1441] Specific operation: Run the Python script and use the analyze_face(image) function to extract and identify features from the facial image.

[1442] Step 6: Interactive question generation and answer collection

[1443] Server operation: The server uses an NLP library to generate interactive questions based on the collected information and sends the questions to the user via the terminal.

[1444] Terminal operation: The terminal displays questions sent from the server and sends answers from the user to the server.

[1445] User action: The user answers questions sent by the server through the terminal.

[1446] Input: Server-generated question, user-entered answer

[1447] Output: The user's answer received by the server

[1448] Specific action: In response to the question "What color clothes do you like?", enter "I like pastel colors" as the answer and submit.

[1449] Step 7: Generate comprehensive diagnostic results

[1450] Server operation: The server integrates the user's profile information, facial image analysis results, and answers to interactive questions, and uses an AI model to generate a comprehensive diagnosis result.

[1451] Input: User profile information, facial image analysis results, answers to dialogue questions

[1452] Output: Comprehensive diagnostic results (suitable fashion items, brands, and retailers)

[1453] Specific operation: Using the TensorFlow model, generate diagnostic results with the generate_recommendations(profile, face_analysis, responses) function.

[1454] Step 8: Send and view results

[1455] Server operation: The server sends the generated diagnostic results to the terminal.

[1456] Device behavior: The device displays the received diagnostic results to the user.

[1457] User Action: User reviews diagnostic results and views suggested items.

[1458] Input: Server-generated diagnostic results

[1459] Output: Diagnostics displayed on the terminal

[1460] How it works: The diagnosis results are displayed on the device screen, and the user can check out the suggested fashion and makeup items.

[1461] In this way, users can quickly and easily find the fashion and makeup items that best suit them through the system.

[1462] (Application example 1)

[1463] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1464] With so many options available, it can be difficult for users to find the right fashion or makeup items. There is also a need for the ability to try on items without actually going to a store. However, the current lack of appropriate technology to make this a reality is problematic. Furthermore, there is a growing need for systems that can dynamically reflect users' preferences and characteristics in real time and make highly accurate recommendations.

[1465] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1466] In this invention, the server includes means for receiving profile information from a user, means for receiving a facial image of the user and analyzing the image to analyze bone structure and personal color, means for interactively asking the user questions and analyzing the user's responses, means for integrating the received profile information, facial image analysis results, and user responses and generating a comprehensive diagnostic result using an AI model, means for suggesting fashion items, brands, and stores suitable for the user based on the generated diagnostic result, and means for capturing images of the user's face and body using smart glasses to provide a virtual try-on experience, allowing the user to try on fashion items in a realistic way without actually going to a store and receiving highly accurate suggestions for fashion items.

[1467] "Profile information" refers to information such as a user's age, gender, and fashion preferences.

[1468] "Facial Image" refers to a photograph or video of a user's face.

[1469] "Image analysis" refers to the process of analyzing received facial images to extract features such as bone structure and personal color.

[1470] An "AI model" refers to an algorithm that uses machine learning or deep learning to analyze data and generate a specific diagnostic result.

[1471] "Comprehensive diagnosis results" refer to results that indicate the fashion items and brands that are best suited to a user, generated by integrating profile information, facial image analysis results, and the user's responses.

[1472] "Dialogue-style questions" refers to a format in which the system asks the user a series of questions and collects the user's answers.

[1473] "Smart glasses" refers to a glasses-type device that, when worn by a user, provides functions such as augmented reality and displays information in the user's field of vision.

[1474] A "virtual try-on experience" is an experience that allows users to get the feeling of trying on clothes in a virtual space without actually trying them on.

[1475] The system embodying this invention utilizes AI to help users find fashion and makeup items that suit them. The specific configuration of the system and its operation method are described below.

[1476] Server Operation

[1477] 1. Receiving profile information

[1478] The server receives the user's profile information (age, gender, fashion preferences, etc.) sent from the device and stores it in a database.

[1479] 2. Receiving and analyzing face images

[1480] After receiving the user's facial image from the device, the server applies image analysis algorithms to analyze the user's bone structure and personal color. The facial image analysis uses OpenCV and Keras models. The analysis results are temporarily stored.

[1481] 3. Interactive Question Generation and Answer Analysis

[1482] The server generates interactive questions and asks them to the user via the device, then uses an AI model to analyze the user's answers and dynamically generate the next questions until the required information is obtained.

[1483] 4. Generating comprehensive diagnostic results

[1484] The server integrates the received profile information, facial image analysis results, and the user's responses, and uses an AI model to generate a comprehensive diagnosis, which includes suitable fashion items, brands, and stores for the user.

[1485] 5. Presentation of results and recommendations

[1486] Based on the generated diagnostic results, the server generates suggestions for specific fashion items, brands, and stores, and transmits them to the terminal to present to the user.

[1487] Virtual try-on experience using smart glasses

[1488] 1. Capture images of your face and body

[1489] The smart glasses capture images of the user's face and body and send them to a server.

[1490] 2. Providing a virtual try-on experience

[1491] Based on the diagnostic results received from the server, the user is given a virtual fitting experience through the smart glasses, allowing them to try on clothes without actually going to a store.

[1492] Specific examples

[1493] Example 1: A woman in her 30s who likes casual fashion

[1494] 1. User: Puts on smart glasses, accesses the app, and enters age (30s), gender (female), and fashion preference (casual).

[1495] 2. Terminal: Sends input information to the server.

[1496] 3. User: Take a photo of their face with smart glasses and upload it.

[1497] 4. Server: Analyzes the facial image and determines that the user is a "spring type." It then generates a dialogue question, asking, "What color clothes do you like?"

[1498] 5. User: "I like pastel colors."

[1499] 6. Server: Based on the answers, identify pastel-colored casual fashion items and suitable brands and generate a diagnosis.

[1500] 7. Terminal: Displays diagnostic results and provides a virtual try-on experience.

[1501] 8. User: Virtually try on the item and decide to purchase from the suggested store.

[1502] Prompt Sentence Examples

[1503] Users enter their age, gender, fashion preferences, and upload a face image. They then answer the following interactive questions:

[1504] 1. What color clothes do you like?

[1505] 2. Where do you go on your days off?

[1506] 3. What style do you like?

[1507] Based on this information, we will display diagnostic results of recommended fashion items, brands, and shops.

[1508] This allows users to have a realistic try-on experience even remotely, and receive highly accurate fashion suggestions.

[1509] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1510] Step 1:

[1511] Enter profile information

[1512] The user puts on the smart glasses and accesses the application, entering their age, gender, and fashion preferences. This profile information is entered into the device and sent to the server.

[1513] Input: Age, Gender, Fashion Preferences

[1514] Output: Profile information sent to the server

[1515] Step 2:

[1516] Enter and send a facial image

[1517] Users use smart glasses to take a picture of their face and upload it to their device, which then sends the image to a server.

[1518] Input: User's face image

[1519] Output: Face image sent to the server

[1520] Step 3:

[1521] Facial image analysis

[1522] The server performs image analysis on the received facial image. This analysis uses OpenCV and Keras models to extract bone structure and personal color features. The analysis results are temporarily saved.

[1523] Input: Face image

[1524] Output: Bone structure and personal color analysis results

[1525] Step 4:

[1526] Interactive question generation

[1527] The server generates interactive questions based on the user's profile information and facial image analysis results, which are then sent to the device and displayed to the user.

[1528] Input: Profile information, facial image analysis results

[1529] Output: Question to the user

[1530] Step 5:

[1531] Receiving and analyzing user responses

[1532] Users answer questions displayed through their devices, and the answers are sent to a server that analyzes the user's answers, using a generative AI model to interpret the meaning of the answers and dynamically generate the next question.

[1533] Input: User's answer

[1534] Output: Parsed answer, next question

[1535] Step 6:

[1536] Generate comprehensive diagnostic results

[1537] The server integrates the received profile information, facial image analysis results, and the user's responses, and uses a generative AI model to generate a comprehensive diagnostic result, which includes suitable fashion items, brands, and stores for the user.

[1538] Input: Profile information, facial image analysis results, user responses

[1539] Output: Overall diagnostic results

[1540] Step 7:

[1541] Displaying diagnostic results

[1542] The server sends the generated diagnostic results to the device, which displays them to the user and prepares for the virtual try-on experience.

[1543] Input: Overall diagnostic results

[1544] Output: Display diagnostic results to the user

[1545] Step 8:

[1546] Providing a virtual try-on experience

[1547] The smart glasses provide users with a virtual try-on experience based on the diagnostic results received from the server, allowing them to try on clothes in a virtual space without actually going to a store.

[1548] Input: Diagnosis results, face and body images from smart glasses

[1549] Output: Virtual try-on experience

[1550] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1551] MODE FOR CARRYING OUT THE INVENTION

[1552] System program and processing description

[1553] The system of this invention utilizes AI and an emotion engine to help users find fashion and makeup items that suit them. The system operates in cooperation with a server, a terminal, and a user, each of which plays a specific role.

[1554] Server Operation

[1555] 1. Receiving profile information:

[1556] The server receives the user's profile information (age, gender, fashion preferences, etc.) sent from the device and stores it in a database.

[1557] 2. Receiving and analyzing face images:

[1558] The server receives the user's facial image from the device, applies an image analysis algorithm to analyze the user's bone structure and personal color, and temporarily stores the facial image analysis results.

[1559] 3. How the Emotion Engine works:

[1560] The server analyzes the user's emotions in real time based on the received facial images and dialogue content, and recognizes their emotional state, which is used in the next processing step.

[1561] 4. Interactive question generation and answer analysis:

[1562] The server generates interactive questions based on the emotion engine, taking into account the user's momentary emotional state, and asks the questions to the user via the device. It receives the user's answers, analyzes their content, and dynamically generates the next questions to ask. This process is repeated until all necessary information is obtained.

[1563] 5. Generating comprehensive diagnostic results:

[1564] The server integrates the received profile information, facial image analysis results, emotional state, and user responses, and uses an AI model to generate a comprehensive diagnostic result, which includes suitable fashion items, brands, and shops for the user.

[1565] 6. Presentation of results and recommendations:

[1566] Based on the generated diagnostic results, the server generates suggestions for specific fashion items, brands, and shops that take into account the emotional state, and transmits them to the terminal to present to the user.

[1567] Device behavior

[1568] 1. Provide profile information input interface:

[1569] The terminal provides an interface for the user to input profile information and transmits the input information to the server.

[1570] 2. Provide face image upload interface:

[1571] The terminal provides an interface for the user to upload a facial image and transmits the uploaded facial image to the server.

[1572] 3. Providing an interactive question and answer input interface:

[1573] The terminal displays the interactive questions sent from the server, provides an interface for the user to answer the questions, and sends the user's answers to the server.

[1574] 4. Provide diagnostic result display interface:

[1575] The terminal provides an interface for displaying the diagnosis results and suggestions sent from the server to the user.

[1576] User behavior

[1577] 1. Enter your profile information:

[1578] Through the device interface, users enter their profile information, including age, gender, and fashion preferences.

[1579] 2. Upload your face image:

[1580] Users upload their facial images using the device's interface.

[1581] 3. Answer the interactive questions:

[1582] The user answers questions sent from the server through the device. For example, to the question "What color clothes do you like?", the user answers "I like pastel colors."

[1583] 4. Review the diagnostic results and accept the recommendations:

[1584] Users can check the diagnostic results displayed on their device and browse suggested fashion items, brands, and shops.

[1585] Specific examples

[1586] Example 1: A woman in her 30s who likes casual fashion

[1587] 1. User: Accesses the app and enters age (30s), gender (female), and fashion preference (casual).

[1588] 2. Terminal: Sends input information to the server.

[1589] 3. User: Upload a photo of your face.

[1590] 4. Device: Sends a photo of your face to the server.

[1591] 5. Server: Performs facial image analysis and determines the person as a "spring type." The analysis results are temporarily saved.

[1592] 6. Server: Uses an emotion engine to recognize emotions from the user's facial images and answers, and understands their current emotional state.

[1593] 7. Server: Generates questions and asks the user, "What color clothes do you like?". Adjusts the questions depending on the user's emotional state.

[1594] 8. Terminal: displays the question and accepts the user's answer.

[1595] 9. User: "I like pastel colors."

[1596] 10. Terminal: Sends the answer to the server.

[1597] 11. Server: Analyzes the answer and generates the next question. Sends "What kind of clothes would you like to wear for what occasion?". Adjusts the next question based on the emotional state.

[1598] 12. Terminal: Show question.

[1599] 13. User: Enter "Relaxing holiday scene."

[1600] 14. Terminal: Sends the answer to the server, analyzes the answer, and repeats this process until all the necessary information is gathered.

[1601] 15. Server: Using an AI model based on facial image analysis results, answers to interactive questions, and emotional state, the server generates diagnostic results suitable for casual fashion, including information on "pastel-colored items" and "suitable brands."

[1602] 16. Server: Dynamically generates diagnostic results and specific recommendations.

[1603] 17. Terminal: Displays the diagnostic results and suggestions to the user.

[1604] 18. User: Based on the presented results, select a pastel-colored casual fashion item and purchase it from the suggested shop.

[1605] In this way, by combining emotion engines, it is possible to provide highly accurate personalized fashion advice according to the user's emotional state.

[1606] The processing flow will be explained below.

[1607] Program processing steps

[1608] Step 1: Enter your user information

[1609] User: Visits an app or website and enters profile information such as age, gender, and fashion preferences.

[1610] Terminal: Receives profile information entered by the user and sends it to the server.

[1611] Server: Stores the received profile information in a database and prepares for the next step.

[1612] Step 2: Upload and analyze face images

[1613] Users: Upload a photo of themselves to an app or website.

[1614] Terminal: Sends the uploaded face photo to the server.

[1615] Server: Analyzes bone structure and personal color using facial image analysis algorithms. Saves the analysis results temporarily and uses them in the next step.

[1616] Step 3: Emotion analysis using the emotion engine

[1617] Server: Analyzes the user's emotions in real time using an emotion engine based on facial images and dialogue content. Temporarily stores the analysis results.

[1618] Step 4: Interactive question generation and answer analysis

[1619] Server: Generates conversational questions that take into account the user's emotional state and sends them to the device. For example, set questions such as "What color clothes do you like?" or "What kind of clothes do you like to wear for what occasions?"

[1620] Terminal: Presents an interface for displaying interactive questions to the user and entering answers.

[1621] User: Enters an answer to a question.

[1622] Terminal: Sends the entered answer to the server.

[1623] Server: Analyzes the received answers and dynamically generates the next question to ask. This process is repeated until all necessary questions have been asked.

[1624] Step 5: Generate comprehensive diagnostic results

[1625] Server: Integrates the results of facial image analysis, answers collected through dialogue questions, and emotional state, and uses AI models to generate comprehensive diagnostic results. This identifies the user's personal color, clothing styles and designs that suit them, popular brands, and suitable accessories and makeup items.

[1626] Server: Stores the diagnostic results in a database and prepares the results for display.

[1627] Step 6: Present your results and make recommendations

[1628] Server: Based on the diagnosis results, it generates recommendations for specific fashion items, brands, and shops and sends them to the device. The recommendations may be updated in real time.

[1629] Terminal: Provides an interface for displaying the diagnostic results and suggestions sent from the server to the user.

[1630] User: Based on the results presented, users can choose fashion and makeup items that suit them best.

[1631] Specific examples

[1632] Example: A woman in her 30s who likes casual fashion

[1633] 1. Step 1:

[1634] User: Accesses the app and enters age (30s), gender (female), and fashion preference (casual).

[1635] Terminal: Sends input information to the server.

[1636] Server: Saves the input information in a database.

[1637] 2. Step 2:

[1638] User: Upload a photo of themselves to the app.

[1639] Device: Sends a photo of your face to the server.

[1640] Server: Performs facial image analysis and determines "spring type." Temporarily saves the analysis results.

[1641] 3. Step 3:

[1642] Server: Uses an emotion engine to recognize emotions from the user's facial image and responses, and understands their current emotional state. For example, it determines whether the user is in a "peace of mind" state.

[1643] 4. Step 4:

[1644] Server: Based on the emotional state, generate a question like "What color clothes do you like?" and send it to the user. The question content is adjusted depending on the emotional state.

[1645] Terminal: Provides an interface for displaying questions and accepting user answers.

[1646] User: "I like pastel colors."

[1647] Terminal: Sends the answer to the server.

[1648] Server: Analyzes the answer and generates the next question: "What kind of occasions do you want to wear your clothes for?" The next question is adjusted taking into account the emotional state.

[1649] Terminal: Display the question.

[1650] User: Enter "Relaxing holiday scene."

[1651] Device: Sends the answer to the server and repeats this process until it has gathered the required information.

[1652] 5. Step 5:

[1653] Server: Using an AI model based on facial image analysis, responses to interactive questions, and emotional state, the server generates a comprehensive diagnosis suitable for casual fashion, including information on "pastel-colored items" and "suitable brands."

[1654] Server: Stores the diagnostic results in a database.

[1655] 6. Step 6:

[1656] Server: Generates diagnostic results and specific fashion item, brand, and shop recommendations based on the results. Suggestions may be updated in real time.

[1657] Terminal: Provides an interface for displaying diagnostic results and recommendations to the user.

[1658] User: Based on the presented results, select a pastel-colored casual fashion item and purchase it from the suggested shop.

[1659] In this way, users can receive highly accurate and realistic fashion advice that takes their emotions into consideration.

[1660] Example 2

[1661] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1662] Conventional fashion advice systems often make suggestions based solely on the user's profile information or image analysis results, and are unable to consider the user's emotional state or real-time changes in preferences. This makes it difficult to provide personalized, highly accurate fashion advice, resulting in reduced user satisfaction. The present invention aims to solve these problems and provide personalized fashion advice that takes the user's emotional state into account.

[1663] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1664] In this invention, the server includes a means for receiving input information from a user, a means for receiving a facial image of the user and analyzing facial features and color characteristics through image processing, and a means for analyzing the user's emotions in real time and dynamically generating questions based on the analysis results. This makes it possible to integrate the user's profile information, facial image analysis results, emotion analysis results, and user answers, use a generative AI model to generate highly accurate comprehensive diagnostic results, and suggest clothing items, brands, and sales locations that are suitable for the user.

[1665] "Input Information" means the personal information and profile data that a User provides to the System.

[1666] A "face image" refers to a photograph or image data of a user's face.

[1667] "Image processing" refers to the technology of analyzing image data and extracting specific features or attributes.

[1668] "Facial features" refers to the shape, bone structure, and feature points of the user's face.

[1669] "Color characteristics" refers to a personal color classification based on the user's face and skin color.

[1670] "Emotion analysis" is a technology that analyzes a user's emotional state in real time from facial images and dialogue content.

[1671] A "generative AI model" is an artificial intelligence model that uses machine learning and natural language processing to analyze data and generate interactive questions and diagnostic results.

[1672] "Comprehensive diagnosis results" are diagnostic results generated by the AI ​​model based on the user's profile information, facial image analysis results, emotion analysis results, and the user's responses.

[1673] "Clothing items" refers to clothes, accessories, etc. suggested to users.

[1674] "Trademark" means the name or logo of a particular brand or manufacturer, which is associated with the proposed fashion item.

[1675] "Point of Sale" refers to a store or online shop where the products suggested to the user can be purchased.

[1676] This invention is a system that utilizes AI and an emotion engine to help users find fashion and makeup items that suit them. The system works in cooperation with a server, a device, and a user, each of which plays a specific role.

[1677] Server Operation

[1678] 1. Receiving profile information

[1679] The server receives the user's input information (age, gender, fashion preferences, etc.) sent from the device and stores it in a database. Specifically, it uses a MySQL database.

[1680] 2. Receiving and analyzing face images

[1681] After receiving the user's facial image from the device, the server processes the image using OpenCV and Dlib. The facial image is analyzed to analyze the user's skeletal and color characteristics, and the analysis results are temporarily stored in Redis.

[1682] 3. Operation of the Emotion Engine

[1683] The server uses Microsoft Azure's emotion analysis API to analyze the user's emotions in real time from facial images and dialogue content, recognizing their emotional state. The analysis results are used in the next step of question generation.

[1684] 4. Question Generation and Answer Analysis

[1685] The server generates questions that take into account the user's temporary emotional state based on the results of sentiment analysis. The generated questions are sent to the user via the device and the user's answers are received. The received answers are then analyzed and the next question is dynamically generated. The NLP technology used is SpaCy and the BERT model.

[1686] 5. Generating comprehensive diagnostic results

[1687] The server integrates the received input information, facial image analysis results, emotional state, and user responses, and uses a generative AI model (e.g., GPT-4) to generate a comprehensive diagnosis result, which includes clothing items, brands, and sales locations suitable for the user.

[1688] 6. Presentation of results and recommendations

[1689] Based on the generated diagnostic results, the server generates specific suggestions that take into account the emotional state, and transmits them to the terminal to present to the user.

[1690] Device behavior

[1691] 1. Provides a profile information input interface

[1692] The terminal provides an interface for users to input information and sends the input information to the server, specifically using JavaScript and ReactJS.

[1693] 2. Providing a face image upload interface

[1694] The terminal provides an interface for the user to upload a facial image and transmits the uploaded facial image to the server.

[1695] 3. Providing a question and answer input interface

[1696] The terminal provides an interface for displaying the questions sent from the server and accepting the user's answers, and then sends the user's answers to the server.

[1697] 4. Providing an interface for displaying diagnostic results

[1698] The terminal provides an interface for displaying the diagnosis results and suggestions sent from the server to the user.

[1699] User behavior

[1700] 1. Enter your profile information

[1701] Through the device's interface, users enter their personal information, including age, gender, and fashion preferences.

[1702] 2. Upload a face image

[1703] Users upload their facial images using the device's interface.

[1704] 3. Answering questions

[1705] The user answers questions sent from the server through the device. For example, to the question "What color clothes do you like?", the user answers "I like pastel colors."

[1706] 4. Review the diagnostic results and accept the recommendations

[1707] The user checks the diagnostic results displayed on the device and browses suggested clothing items, brands, and sales locations.

[1708] Specific examples

[1709] Example 1: A woman in her 30s who likes casual fashion

[1710] 1. User: Accesses the app and enters age (30s), gender (female), and fashion preference (casual).

[1711] 2. Terminal: Sends input information to the server.

[1712] 3. Server: Stores profile information in a MySQL database.

[1713] 4. User: Upload a photo of your face.

[1714] 5. Device: Sends a photo of your face to the server.

[1715] 6. Server: Performs facial image analysis and determines "spring type." Temporarily stores the analysis results in Redis.

[1716] 7. Server: Uses an emotion engine to recognize emotions from the user's facial images and answers, and understands their current emotional state.

[1717] 8. Server: Generates questions and asks the user, "What color clothes do you like?". Adjusts the questions depending on the user's emotional state.

[1718] 9. Terminal: displays the question and accepts the user's answer.

[1719] 10. User: "I like pastel colors."

[1720] 11. Terminal: Sends the answer to the server.

[1721] 12. Server: Analyzes the answer and generates the next question. Sends "What kind of clothes would you like to wear for what occasion?". Adjusts the next question based on the emotional state.

[1722] 13. Terminal: Display the question.

[1723] 14. User: Enter "Relaxing holiday scene."

[1724] 15. Terminal: Sends the answer to the server, analyzes the answer, and repeats this process until all the necessary information is gathered.

[1725] 16. Server: Using a generative AI model based on facial image analysis results, answers to interactive questions, and emotional state, the server generates diagnostic results suitable for casual fashion, including information on "pastel-colored items" and "suitable brands."

[1726] 17. Server: Dynamically generates diagnostic results and specific recommendations.

[1727] 18. Terminal: Display the diagnostic results and suggestions to the user.

[1728] 19. User: Based on the presented results, select a pastel-colored casual fashion item and purchase it at the suggested retail location.

[1729] Prompt Sentence Examples

[1730] "I'm a woman in my 30s who likes casual fashion. My face image indicates I'm a spring type. What fashion items and brands would you suggest?"

[1731] In this way, it is possible to provide personalized fashion advice according to the user's emotional state.

[1732] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1733] Step 1:

[1734] Enter profile information

[1735] The user enters their own information (age, gender, fashion preferences, etc.) through the device interface. The device validates the entered information using JavaScript and sends it to the server. The server receives the input information and saves it in a MySQL database using INSERT statements. The input is the user's profile information, and the output is the profile information saved in the database.

[1736] Step 2:

[1737] Upload a face image

[1738] A user uploads their facial image using the device interface. The device encodes the uploaded facial image into Base64 format and sends it to the server. The server receives the facial image and temporarily stores it in a file system or database. The input is the user's facial image, and the output is the facial image stored on the server.

[1739] Step 3:

[1740] Facial image analysis

[1741] The server analyzes the stored facial images using OpenCV and Dlib. First, it uses a face detection algorithm to extract facial feature points, then analyzes the bone structure and color characteristics. The analysis results are temporarily stored in Redis. The input is the user's facial image, and the output is the analysis results including bone structure and color characteristics.

[1742] Step 4:

[1743] Emotion analysis

[1744] The server uses Microsoft Azure's emotion analysis API to analyze the user's emotions in real time from facial images and dialogue content. It makes API calls, obtains the analysis results, and stores them in a database. The input is the user's facial image and dialogue content, and the output is the analysis results that indicate the user's emotional state.

[1745] Step 5:

[1746] Question generation and answer analysis

[1747] The server takes into account the user's emotional state based on the sentiment analysis results and uses a generative AI model to dynamically generate the next question. It then sends the generated question to the user via their device and receives the user's answer. It analyzes the answer and, if necessary, generates the next question and repeats the process. The input is the sentiment analysis result and the user's answer, and the output is the dynamically generated next question.

[1748] Step 6:

[1749] Generate comprehensive diagnostic results

[1750] The server integrates the profile information, facial image analysis results, emotion analysis results, and the user's responses, and uses a generative AI model to generate a comprehensive diagnosis. Specifically, a prompt is entered to query the AI ​​model and obtain the results. The diagnosis results include clothing items, trademarks, and sales locations suitable for the user. The input is the integrated user information, and the output is the comprehensive diagnosis results.

[1751] Step 7:

[1752] Presentation of results and recommendations

[1753] The server sends the generated diagnostic results to the terminal. The terminal provides an interface to display the diagnostic results and suggestions to the user. The user can review the presented results and select items that suit their preferences. The input is the overall diagnostic results, and the output is the suggestions displayed to the user.

[1754] (Application example 2)

[1755] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1756] Conventional fashion and makeup item recommendation systems have issues with insufficient suggestions based on user profile information or facial image analysis, and are unable to provide personalized advice that takes into account the user's emotional state. Furthermore, they lack the ability to dynamically generate questions that respond to the user's preferences and emotions, or to adjust the recommendations in real time, making it difficult to recommend optimal products to users.

[1757] The identification processing by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for receiving profile information from a user, means for receiving a facial image of the user and analyzing the user's bone structure and personal color through image analysis, means for interactively asking the user questions and analyzing the user's responses, means for integrating the received profile information, facial image analysis results, and the user's responses and generating a comprehensive diagnostic result using an AI model, means for proposing products suitable for the user based on the diagnostic result, means for analyzing the user's emotional state in real time and dynamically adjusting the content of the interactive questions based on the emotional state, and means for inputting prompt sentences into a generative AI model generated based on the emotional state and profile information to make personalized suggestions. This enables optimal product suggestions in real time while taking the user's emotional state into consideration.

[1758] "Profile Information" is information about a user's personal information, such as their age, gender, and fashion preferences.

[1759] "Bone structure" refers to features that represent the structure of a user's face and body.

[1760] "Personal Color" refers to the range of colors that best complement a user's natural skin tone and hair color.

[1761] "Dialogue" is a format in which the user and the system collect information by exchanging questions and answers.

[1762] An "AI model" is a mathematical model that uses machine learning and artificial intelligence techniques to analyze data and make recommendations and predictions.

[1763] The "comprehensive diagnosis result" is the result of optimal suggestions for the user, generated based on the received profile information, facial image analysis results, and user responses.

[1764] "Emotional state" indicates the user's current emotional state and is estimated from facial expressions and dialogue content.

[1765] A "generative AI model" is an AI model that generates personalized suggestions based on a user's profile information and emotional state using specific prompt sentences as input.

[1766] A "prompt sentence" is text that is input into a generative AI model and serves as the basis for generating specific suggestions for the user.

[1767] "Dynamic adjustment" means making changes and adaptations instantly in response to the user's real-time reactions and conditions.

[1768] This invention relates to a system that enables users to find fashion items and makeup items that suit them, and is mainly comprised of a server, a terminal, and a user, each of which plays a specific role and operates in cooperation with one another.

[1769] System configuration

[1770] Server Operation

[1771] 1. The server receives the user's profile information (age, gender, fashion preferences, etc.) sent from the device and stores it in a database.

[1772] 2. After receiving the user's facial image from the device, the server applies image analysis algorithms to analyze the facial structure and personal color. This image analysis uses image processing libraries such as OpenCV.

[1773] 3. The server uses the Emotion Engine to analyze the user's emotions in real time based on the received facial images and dialogue content, and recognizes their emotional state.

[1774] 4. The server generates dialogue-style questions based on the emotion engine, taking into account the user's temporary emotional state, and asks the questions to the user via the terminal. This can be done using a framework such as Flask or Django.

[1775] 5. The server integrates the received profile information, facial image analysis results, emotional state, and the user's answers, and generates a comprehensive diagnosis result using an AI model called AIFashionModel, which includes fashion items, brands, and shops suitable for the user.

[1776] 6. The server generates suggestions for specific fashion items, brands, and shops that take into account the emotional state, and sends them to the terminal to present to the user.

[1777] Device behavior

[1778] 1. The terminal provides an interface for the user to input profile information and transmits the input information to the server.

[1779] 2. The terminal provides an interface for the user to upload a facial image and transmits the uploaded facial image to the server.

[1780] 3. The terminal displays the interactive questions sent from the server, provides an interface for the user to answer the questions, and sends the user's answers to the server.

[1781] 4. The terminal provides an interface for displaying the diagnostic results and suggestions sent from the server to the user.

[1782] User behavior

[1783] 1. Through the device interface, users enter their profile information, including age, gender, and fashion preferences.

[1784] 2. The user uploads an image of their face using the device interface.

[1785] 3. The user answers questions sent from the server through their device. For example, to the question "What color clothes do you like?", the user answers "I like pastel colors."

[1786] 4. The user checks the diagnostic results displayed on the device and browses suggested fashion items, brands, and shops.

[1787] Specific examples

[1788] For example, for a female user in her 30s who likes casual fashion,

[1789] 1. The user enters their age (30s), gender (female), and fashion preference (casual).

[1790] 2. Upload a photo of your face and your face will be analyzed to determine your "spring type."

[1791] 3. The server uses an emotion engine to recognize the user's "current emotional state" from the facial image and generates interactive questions.

[1792] 4. For example, ask the user, "What color clothes do you like?"

[1793] 5. The user answers, "I like pastel colors," and based on this, the next question is dynamically generated, such as, "What kind of clothes do you like to wear for what occasions?"

[1794] 6. This process is repeated until all the necessary information is gathered, and finally, personalized fashion items are suggested.

[1795] Prompt Sentence Examples

[1796] If the profile information is "30s, female, casual fashion" and the emotional state is "happy," the server will input the following prompt sentence to the AI ​​model:

[1797] "A woman in her 30s likes casual fashion and spring-type pastel colors. She is currently in a happy emotional state. Please suggest some fashion items that would be perfect for relaxing on the weekend."

[1798] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1799] Step 1:

[1800] Users enter profile information through the device interface, such as age, gender, and fashion preferences. This information becomes input data, which the device receives and transmits to the server.

[1801] Step 2:

[1802] The server stores the user profile information received from the device in a database, which will be referenced in subsequent processing.

[1803] Step 3:

[1804] The user takes a picture of their face using the device's camera and uploads it to the server. The face image becomes input data, and the device receives this face image data and sends it to the server.

[1805] Step 4:

[1806] The server analyzes the received facial images using an image processing library such as OpenCV. This analysis identifies the facial bone structure and personal color, which are then temporarily saved as intermediate data.

[1807] Step 5:

[1808] The server uses EmotionEngine to analyze the user's emotional state in real time based on the received facial images and dialogue content. The emotional state is output as an analysis result and used in the next step.

[1809] Step 6:

[1810] Based on the analyzed emotional state, the server dynamically generates interactive questions, which are tailored to the user's current emotions and sent to the device as question data.

[1811] Step 7:

[1812] The terminal presents the user with interactive questions sent from the server, and the user answers them, and the terminal receives the answer data and sends it to the server.

[1813] Step 8:

[1814] The server analyzes the user's answers and dynamically generates subsequent questions to obtain further required information. This process is repeated until the required information is gathered.

[1815] Step 9:

[1816] The server integrates the received profile information, facial image analysis results, emotional state, and user responses, and generates a comprehensive diagnosis result using the AIFashionModel, which includes a list of products suitable for the user, and generates this data as recommendation data.

[1817] Step 10:

[1818] The server takes into account the emotional state and diagnosis results and inputs the optimal suggestion into the generative AI model as a prompt sentence, which contains specific fashion item suggestions for the user.

[1819] Step 11:

[1820] The generated suggestions are sent to the device and presented to the user. The user can use these suggestions as a reference when selecting products. For example, a prompt might read, "A woman in her 30s likes casual fashion and spring-type pastel colors. Her current emotional state is happy. Please suggest the best fashion items for a relaxing holiday."

[1821] This makes it possible to provide personalized fashion advice and suggestions that take into account emotional state in real time.

[1822] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1823] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1824] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[1825] [Fourth embodiment]

[1826] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[1827] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1828] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1829] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[1830] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1831] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1832] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1833] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[1834] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1835] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1836] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1837] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1838] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1839] MODE FOR CARRYING OUT THE INVENTION

[1840] System program and processing description

[1841] The system of this invention utilizes AI to help users find fashion and makeup items that suit them. The system operates in cooperation with a server, a terminal, and a user, each of which plays a specific role.

[1842] Server Operation

[1843] 1. Receiving profile information:

[1844] The server receives the user's profile information (age, gender, fashion preferences, etc.) sent from the device and stores it in a database.

[1845] 2. Receiving and analyzing face images:

[1846] The server receives the user's facial image from the device, applies an image analysis algorithm to analyze the user's bone structure and personal color, and temporarily stores the facial image analysis results.

[1847] 3. Interactive question generation and answer analysis:

[1848] The server generates interactive questions and asks them to the user via the terminal, receives the user's answers, analyzes the answers, and dynamically generates the next questions to ask. This process is repeated until all necessary information is obtained.

[1849] 4. Generating comprehensive diagnostic results:

[1850] The server integrates the received profile information, facial image analysis results, and user responses, and uses an AI model to generate a comprehensive diagnosis, which includes suitable fashion items, brands, and shops for the user.

[1851] 5. Presentation of results and recommendations:

[1852] Based on the generated diagnostic results, the server generates suggestions for specific fashion items, brands, and shops, and transmits them to the terminal to present to the user.

[1853] Device behavior

[1854] 1. Provide profile information input interface:

[1855] The terminal provides an interface for the user to input profile information and transmits the input information to the server.

[1856] 2. Provide face image upload interface:

[1857] The terminal provides an interface for the user to upload a facial image and transmits the uploaded facial image to the server.

[1858] 3. Providing an interactive question and answer input interface:

[1859] The terminal displays the questions sent from the server, provides an interface for the user to answer the questions, and sends the user's answers to the server.

[1860] 4. Provide diagnostic result display interface:

[1861] The terminal provides an interface for displaying the diagnosis results and suggestions sent from the server to the user.

[1862] User behavior

[1863] 1. Enter your profile information:

[1864] Through the device interface, users enter their profile information, including age, gender, and fashion preferences.

[1865] 2. Upload your face image:

[1866] Users upload their facial images using the device's interface.

[1867] 3. Answer the interactive questions:

[1868] The user answers questions sent from the server through the device. For example, to the question "What color clothes do you like?", the user answers "I like pastel colors."

[1869] 4. Review the diagnostic results and accept the recommendations:

[1870] Users can check the diagnostic results displayed on their device and browse suggested fashion items, brands, and shops.

[1871] Specific examples

[1872] Example 1: A woman in her 30s who likes casual fashion

[1873] 1. User: Accesses the app and enters age (30s), gender (female), and fashion preference (casual).

[1874] 2. Terminal: Sends input information to the server.

[1875] 3. User: Upload a photo of your face.

[1876] 4. Device: Send a photo of your face to the server.

[1877] 5. Server: Performs facial image analysis and determines the user is a "spring type." Generates a dialogue question, asking, "What color clothes do you like?"

[1878] 6. User: "I like pastel colors."

[1879] 7. Terminal: Sends the answer to the server.

[1880] 8. Server: Based on all the information, identify pastel-colored casual fashion items and suitable brands and generate a diagnosis.

[1881] 9. Terminal: Display the diagnostic results.

[1882] 10. User: Select an item based on the diagnosis results and purchase it from the suggested shop.

[1883] Example 2: A man in his 40s who likes formal fashion

[1884] 1. User: Accesses the app and enters age (40s), gender (male), and fashion preference (formal).

[1885] 2. Terminal: Sends input information to the server.

[1886] 3. User: Upload a photo of your face.

[1887] 4. Device: Send a photo of your face to the server.

[1888] 5. Server: Performs facial image analysis and determines the person is a "winter type." Generates a dialogue question, asking, "What color do you prefer for formal occasions?"

[1889] 6. User: "Dark blue or black."

[1890] 7. Terminal: Sends the answer to the server.

[1891] 8. Server: Based on all the information, it identifies formal fashion items and popular brands and generates diagnostic results.

[1892] 9. Terminal: Display the diagnostic results.

[1893] 10. User: Select an item based on the diagnosis results and purchase it from the suggested shop.

[1894] In this way, users can easily find the fashion style that suits them through highly accurate diagnosis. The system handles everything from entering profile information, analyzing facial images, asking interactive questions, and generating and proposing comprehensive diagnosis results.

[1895] The processing flow will be explained below.

[1896] Program processing steps

[1897] Step 1: Enter your user information

[1898] User: Visits an app or website and enters profile information such as age, gender, and fashion preferences.

[1899] Terminal: Receives profile information entered by the user and sends it to the server.

[1900] Server: Stores the received profile information in a database and prepares for the next step.

[1901] Step 2: Upload and analyze face images

[1902] Users: Upload a photo of themselves to an app or website.

[1903] Terminal: Sends the uploaded face photo to the server.

[1904] Server: Analyzes bone structure and personal color using facial image analysis algorithms. Saves the analysis results temporarily and uses them in the next step.

[1905] Step 3: Interactive question generation and answer analysis

[1906] Server: Generates interactive questions and sends them to the device. Initial questions include "What color clothes do you like?" and "What kind of clothes do you like to wear for what occasions?"

[1907] Terminal: Provides an interface for displaying interactive questions to the user and for entering answers.

[1908] User: Enters an answer to a question.

[1909] Terminal: Sends the entered answer to the server.

[1910] Server: Analyzes the received answers and dynamically generates the next question to ask. This process is repeated until all necessary questions have been asked.

[1911] Step 4: Generate comprehensive diagnostic results

[1912] Server: The server combines the results of facial image analysis with the user's responses to interactive questions and uses an AI model to generate a comprehensive diagnosis, specifically identifying the user's personal color, clothing styles and designs that suit them, popular brands, and suitable accessories and makeup items.

[1913] Server: Stores the diagnostic results in a database and prepares the results for display.

[1914] Step 5: Present the results and make recommendations

[1915] Server: Generates the diagnosis results and specific fashion item, brand, and shop recommendations based on them. The recommendations may be updated in real time, so they must be generated dynamically.

[1916] Terminal: Provides an interface for presenting diagnostic results to the user.

[1917] User: Based on the results presented, users can choose fashion and makeup items that suit them best.

[1918] Specific examples

[1919] Example: A woman in her 30s who likes casual fashion

[1920] 1. Step 1:

[1921] User: Accesses the app and enters age (30s), gender (female), and fashion preference (casual).

[1922] Terminal: Sends input information to the server.

[1923] Server: Saves the input information in a database.

[1924] 2. Step 2:

[1925] User: Upload a photo of themselves to the app.

[1926] Device: Sends a photo of your face to the server.

[1927] Server: Performs facial image analysis and determines "spring type." Temporarily saves the analysis results.

[1928] 3. Step 3:

[1929] Server: Generates a dialogue question and asks the user, "What color clothes do you like?"

[1930] Terminal: Displays questions and accepts user answers.

[1931] User: "I like pastel colors."

[1932] Terminal: Sends the answer to the server.

[1933] Server: Analyzes the answer and generates the next question: "What kind of clothes would you like to wear for what occasion?"

[1934] Terminal: Show question.

[1935] User: Enter "Relaxing holiday scene."

[1936] Device: Sends the answer to the server, analyzes the answer, and repeats this process until it has gathered all the information it needs.

[1937] 4. Step 4:

[1938] Server: Using an AI model based on the results of facial image analysis and answers to interactive questions, the server generates diagnostic results suitable for casual fashion, including information on "pastel-colored items" and "suitable brands."

[1939] Server: Stores the diagnostic results in a database.

[1940] 5. Step 5:

[1941] Server: Dynamically generates diagnostic results and recommendations for specific fashion items, brands, and shops based on those results.

[1942] On the device: Display diagnostic results and suggestions to the user.

[1943] User: Based on the presented results, select a pastel-colored casual fashion item and purchase it from the suggested shop.

[1944] In this way, users can receive highly accurate and realistic fashion advice throughout the process.

[1945] Example 1

[1946] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1947] Users often spend a lot of time and effort finding the perfect fashion and makeup items for themselves, and it can be particularly difficult to choose items that take into account facial features and personal color. Current systems are unable to accurately analyze these points and make real-time suggestions that match the user's preferences and characteristics. This leaves users with the challenge of being unable to quickly and easily find the perfect fashion and makeup items based on their preferences and characteristics.

[1948] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1949] In this invention, the server includes means for receiving profile information from a user, means for receiving a facial image of the user and analyzing bone structure and personal color through image analysis, means for interactively asking the user questions and analyzing the user's responses, means for integrating the received profile information, facial image analysis results, and user responses and generating a comprehensive diagnostic result using an artificial intelligence model, means for suggesting clothing items, brands, and retailers suitable for the user based on the diagnostic result, communication means for sending initial information to the server in JSON format, means for sending a facial image via an HTTP request, means for analyzing the facial image using an image analysis library, means for dynamically generating interactive questions using the profile information and image analysis results, and means for generating and presenting a comprehensive diagnostic result, thereby enabling users to quickly and easily find fashion and makeup items based on their characteristics and preferences.

[1950] "Profile information" refers to information about a user's personal characteristics, such as the user's age, gender, and fashion preferences.

[1951] A "face image" is an image of the user's face, and facial features and personal color are analyzed based on this image.

[1952] "Image analysis" is a process performed on a received facial image to identify the user's bone structure and personal color.

[1953] "Dialogue format" refers to a format in which questions are asked to the user sequentially, and the next question is dynamically generated based on the answers.

[1954] An "artificial intelligence model" is a model that uses technologies such as machine learning and deep learning, and generates comprehensive diagnostic results based on the received data.

[1955] The "comprehensive diagnosis results" are generated by integrating profile information, facial image analysis results, and user responses, and include suggestions for fashion items, trademarks, and retailers that are suitable for the user.

[1956] "Communication means" refers to the technical means for transmitting initial information and analysis data to the server in JSON format or via HTTP requests.

[1957] The "JSON format" is a format that structures and expresses data in text format, and is primarily used for sending and receiving data.

[1958] An "HTTP request" is a protocol that allows a client to request a server to send data or obtain information.

[1959] An "image analysis library" is a software library, such as OpenCV or Dlib, that is used to extract and analyze features from facial images.

[1960] "Dynamic generation" means adaptively generating the next question or suggestion on the spot based on the user's answers.

[1961] "Diagnosis results" are recommendations for fashion items and services suitable for the user, generated based on analysis of the received data and artificial intelligence models.

[1962] MODE FOR CARRYING OUT THE INVENTION

[1963] This invention provides a system in which a server, a terminal, and a user work together to help users find the fashion and makeup items that are best suited to them. The overall configuration of the system and the operation of each component are described in detail below.

[1964] Server Operation

[1965] 1. Receiving and storing profile information:

[1966] The server receives the user's profile information (age, gender, fashion preferences, etc.) sent from the device and stores it in a database (e.g., MySQL or PostgreSQL), making it possible to manage each user's individual information.

[1967] 2. Receiving and analyzing face images:

[1968] After receiving the user's facial image from the device, the server analyzes the user's bone structure and personal color using image analysis libraries such as OpenCV and Dlib. The analysis results are temporarily stored in a database.

[1969] 3. Interactive question generation and answer analysis:

[1970] The server uses an NLP library (e.g., NLTK or SpaCy) to generate interactive questions and pose them to the user through the terminal. After receiving the user's answers, it analyzes their content and dynamically generates the next questions.

[1971] 4. Generating comprehensive diagnostic results:

[1972] The server integrates the profile information, facial image analysis results, and user responses, and generates a comprehensive diagnostic result using an AI model (e.g., TensorFlow or PyTorch), which can identify suitable fashion and makeup items for the user.

[1973] 5. Presentation of results and recommendations:

[1974] The server generates specific suggestions based on the generated diagnostic results and sends them to the terminal for presentation to the user, including clothing items, brands, and retailers suitable for the user.

[1975] Device behavior

[1976] 1. Provide profile information input interface:

[1977] The terminal provides an interface for the user to input profile information and transmits the input information to the server.

[1978] 2. Provide face image upload interface:

[1979] The terminal provides an interface for the user to upload a facial image and transmits the uploaded facial image to the server.

[1980] 3. Providing an interactive question and answer input interface:

[1981] The terminal displays the questions sent from the server, provides an interface for the user to answer the questions, and sends the user's answers to the server.

[1982] 4. Provide diagnostic result display interface:

[1983] The terminal provides an interface for displaying the diagnosis results and suggestions sent from the server to the user.

[1984] User behavior

[1985] 1. Enter your profile information:

[1986] Users enter their profile information through the device interface, including their age, gender and fashion preferences.

[1987] 2. Upload your face image:

[1988] Users upload their facial images using the device's interface.

[1989] 3. Answer the interactive questions:

[1990] The user answers questions sent from the server through the device. For example, to the question "What color clothes do you like?", the user answers "I like pastel colors."

[1991] 4. Review the diagnostic results and accept the recommendations:

[1992] Users can check the diagnosis results displayed on their device and use the suggested fashion and makeup items as a reference.

[1993] Specific use cases

[1994] Example 1: A woman in her 30s who likes casual fashion

[1995] 1. User: Accesses the app and enters age (30s), gender (female), and fashion preference (casual).

[1996] 2. Terminal: Sends input information to the server.

[1997] 3. User: Upload a photo of yourself.

[1998] 4. Device: Sends a photo of your face to the server.

[1999] 5. Server: Performs facial image analysis and determines the user is a "spring type." Generates a dialogue question, asking, "What color clothes do you like?"

[2000] 6. User: "I like pastel colors."

[2001] 7. Terminal: Sends the answer to the server.

[2002] 8. Server: Based on all the information, it identifies pastel-colored casual fashion items and suitable trademarks and generates a diagnosis result.

[2003] 9. Terminal: Display the diagnostic results.

[2004] 10. User: Select an item based on the diagnosis results and purchase it from the suggested retailer.

[2005] Example 2: A man in his 40s who likes formal fashion

[2006] 1. User: Accesses the app and enters age (40s), gender (male), and fashion preference (formal).

[2007] 2. Terminal: Sends input information to the server.

[2008] 3. User: Upload a photo of yourself.

[2009] 4. Device: Sends a photo of your face to the server.

[2010] 5. Server: Performs facial image analysis and determines the person is a "winter type." Generates a dialogue question, asking, "What color do you prefer for formal occasions?"

[2011] 6. User: "I like dark blue and black."

[2012] 7. Terminal: Sends the answer to the server.

[2013] 8. Server: Based on all the information, it identifies formal fashion items and popular brands and generates diagnostic results.

[2014] 9. Terminal: Display the diagnostic results.

[2015] 10. User: Select an item based on the diagnosis results and purchase it from the suggested retailer.

[2016] In this way, users can quickly and easily find the fashion and makeup items that best suit them through the system.

[2017] The flow of the identification process in the first embodiment will be described with reference to FIG.

[2018] Step 1: Enter your profile information

[2019] User Action: The user enters their profile information (age, gender, fashion preferences) into the interface provided by the device.

[2020] Input: User profile information (e.g., "30s," "Female," "Casual")

[2021] Output: Sending profile information from device to server

[2022] Specific action: Enter data into a form in a web application and press the "Submit" button.

[2023] Step 2: Submit your profile information

[2024] Device behavior: The device sends the entered profile information to the server in JSON format.

[2025] Input: Profile information entered by the user

[2026] Output: Profile information received by the server

[2027] What it does: Sends profile information to the / api / profile endpoint using an HTTP POST request.

[2028] Step 3: Upload your face image

[2029] User action: The user selects and uploads a face image using the device interface.

[2030] Input: A face image selected by the user

[2031] Output: Sending face image from device to server

[2032] Specific operation: Select an image file and press the "Upload" button.

[2033] Step 4: Send a face image

[2034] Device operation: The device sends the uploaded facial image to the server.

[2035] Input: Face image file uploaded by the user

[2036] Output: Face image received by the server

[2037] What it does: Sends an image file to the / api / upload endpoint using an HTTP POST request.

[2038] Step 5: Facial image analysis

[2039] Server operation: The server analyzes the received facial image using an image analysis library such as OpenCV or Dlib to identify bone structure and personal color.

[2040] Input: Face image sent to the server

[2041] Output: Analysis results of bone structure and personal color

[2042] Specific operation: Run the Python script and use the analyze_face(image) function to extract and identify features from the facial image.

[2043] Step 6: Interactive question generation and answer collection

[2044] Server operation: The server uses an NLP library to generate interactive questions based on the collected information and sends the questions to the user via the terminal.

[2045] Terminal operation: The terminal displays questions sent from the server and sends answers from the user to the server.

[2046] User action: The user answers questions sent by the server through the terminal.

[2047] Input: Server-generated question, user-entered answer

[2048] Output: The user's answer received by the server

[2049] Specific action: In response to the question "What color clothes do you like?", enter "I like pastel colors" as the answer and submit.

[2050] Step 7: Generate comprehensive diagnostic results

[2051] Server operation: The server integrates the user's profile information, facial image analysis results, and answers to interactive questions, and uses an AI model to generate a comprehensive diagnosis result.

[2052] Input: User profile information, facial image analysis results, answers to dialogue questions

[2053] Output: Comprehensive diagnostic results (suitable fashion items, brands, and retailers)

[2054] Specific operation: Using the TensorFlow model, generate diagnostic results with the generate_recommendations(profile, face_analysis, responses) function.

[2055] Step 8: Send and view results

[2056] Server operation: The server sends the generated diagnostic results to the terminal.

[2057] Device behavior: The device displays the received diagnostic results to the user.

[2058] User Action: User reviews diagnostic results and views suggested items.

[2059] Input: Server-generated diagnostic results

[2060] Output: Diagnostics displayed on the terminal

[2061] How it works: The diagnosis results are displayed on the device screen, and the user can check out the suggested fashion and makeup items.

[2062] In this way, users can quickly and easily find the fashion and makeup items that best suit them through the system.

[2063] (Application example 1)

[2064] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[2065] With so many options available, it can be difficult for users to find the right fashion or makeup items. There is also a need for the ability to try on items without actually going to a store. However, the current lack of appropriate technology to make this a reality is problematic. Furthermore, there is a growing need for systems that can dynamically reflect users' preferences and characteristics in real time and make highly accurate recommendations.

[2066] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[2067] In this invention, the server includes means for receiving profile information from a user, means for receiving a facial image of the user and analyzing the image to analyze bone structure and personal color, means for interactively asking the user questions and analyzing the user's responses, means for integrating the received profile information, facial image analysis results, and user responses and generating a comprehensive diagnostic result using an AI model, means for suggesting fashion items, brands, and stores suitable for the user based on the generated diagnostic result, and means for capturing images of the user's face and body using smart glasses to provide a virtual try-on experience, allowing the user to try on fashion items in a realistic way without actually going to a store and receiving highly accurate suggestions for fashion items.

[2068] "Profile information" refers to information such as a user's age, gender, and fashion preferences.

[2069] "Facial Image" refers to a photograph or video of a user's face.

[2070] "Image analysis" refers to the process of analyzing received facial images to extract features such as bone structure and personal color.

[2071] An "AI model" refers to an algorithm that uses machine learning or deep learning to analyze data and generate a specific diagnostic result.

[2072] "Comprehensive diagnosis results" refer to results that indicate the fashion items and brands that are best suited to a user, generated by integrating profile information, facial image analysis results, and the user's responses.

[2073] "Dialogue-style questions" refers to a format in which the system asks the user a series of questions and collects the user's answers.

[2074] "Smart glasses" refers to a glasses-type device that, when worn by a user, provides functions such as augmented reality and displays information in the user's field of vision.

[2075] A "virtual try-on experience" is an experience that allows users to get the feeling of trying on clothes in a virtual space without actually trying them on.

[2076] The system embodying this invention utilizes AI to help users find fashion and makeup items that suit them. The specific configuration of the system and its operation method are described below.

[2077] Server Operation

[2078] 1. Receiving profile information

[2079] The server receives the user's profile information (age, gender, fashion preferences, etc.) sent from the device and stores it in a database.

[2080] 2. Receiving and analyzing face images

[2081] After receiving the user's facial image from the device, the server applies image analysis algorithms to analyze the user's bone structure and personal color. The facial image analysis uses OpenCV and Keras models. The analysis results are temporarily stored.

[2082] 3. Interactive Question Generation and Answer Analysis

[2083] The server generates interactive questions and asks them to the user via the device, then uses an AI model to analyze the user's answers and dynamically generate the next questions until the required information is obtained.

[2084] 4. Generating comprehensive diagnostic results

[2085] The server integrates the received profile information, facial image analysis results, and the user's responses, and uses an AI model to generate a comprehensive diagnosis, which includes suitable fashion items, brands, and stores for the user.

[2086] 5. Presentation of results and recommendations

[2087] Based on the generated diagnostic results, the server generates suggestions for specific fashion items, brands, and stores, and transmits them to the terminal to present to the user.

[2088] Virtual try-on experience using smart glasses

[2089] 1. Capture images of your face and body

[2090] The smart glasses capture images of the user's face and body and send them to a server.

[2091] 2. Providing a virtual try-on experience

[2092] Based on the diagnostic results received from the server, the user is given a virtual fitting experience through the smart glasses, allowing them to try on clothes without actually going to a store.

[2093] Specific examples

[2094] Example 1: A woman in her 30s who likes casual fashion

[2095] 1. User: Puts on smart glasses, accesses the app, and enters age (30s), gender (female), and fashion preference (casual).

[2096] 2. Terminal: Sends input information to the server.

[2097] 3. User: Take a photo of their face with smart glasses and upload it.

[2098] 4. Server: Analyzes the facial image and determines that the user is a "spring type." It then generates a dialogue question, asking, "What color clothes do you like?"

[2099] 5. User: "I like pastel colors."

[2100] 6. Server: Based on the answers, identify pastel-colored casual fashion items and suitable brands and generate a diagnosis.

[2101] 7. Terminal: Displays diagnostic results and provides a virtual try-on experience.

[2102] 8. User: Virtually try on the item and decide to purchase from the suggested store.

[2103] Prompt Sentence Examples

[2104] Users enter their age, gender, fashion preferences, and upload a face image. They then answer the following interactive questions:

[2105] 1. What color clothes do you like?

[2106] 2. Where do you go on your days off?

[2107] 3. What style do you like?

[2108] Based on this information, we will display diagnostic results of recommended fashion items, brands, and shops.

[2109] This allows users to have a realistic try-on experience even remotely, and receive highly accurate fashion suggestions.

[2110] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[2111] Step 1:

[2112] Enter profile information

[2113] The user puts on the smart glasses and accesses the application, entering their age, gender, and fashion preferences. This profile information is entered into the device and sent to the server.

[2114] Input: Age, Gender, Fashion Preferences

[2115] Output: Profile information sent to the server

[2116] Step 2:

[2117] Enter and send a facial image

[2118] Users use smart glasses to take a picture of their face and upload it to their device, which then sends the image to a server.

[2119] Input: User's face image

[2120] Output: Face image sent to the server

[2121] Step 3:

[2122] Facial image analysis

[2123] The server performs image analysis on the received facial image. This analysis uses OpenCV and Keras models to extract bone structure and personal color features. The analysis results are temporarily saved.

[2124] Input: Face image

[2125] Output: Bone structure and personal color analysis results

[2126] Step 4:

[2127] Interactive question generation

[2128] The server generates interactive questions based on the user's profile information and facial image analysis results, which are then sent to the device and displayed to the user.

[2129] Input: Profile information, facial image analysis results

[2130] Output: Question to the user

[2131] Step 5:

[2132] Receiving and analyzing user responses

[2133] Users answer questions displayed through their devices, and the answers are sent to a server that analyzes the user's answers, using a generative AI model to interpret the meaning of the answers and dynamically generate the next question.

[2134] Input: User's answer

[2135] Output: Parsed answer, next question

[2136] Step 6:

[2137] Generate comprehensive diagnostic results

[2138] The server integrates the received profile information, facial image analysis results, and the user's responses, and uses a generative AI model to generate a comprehensive diagnostic result, which includes suitable fashion items, brands, and stores for the user.

[2139] Input: Profile information, facial image analysis results, user responses

[2140] Output: Overall diagnostic results

[2141] Step 7:

[2142] Displaying diagnostic results

[2143] The server sends the generated diagnostic results to the device, which displays them to the user and prepares for the virtual try-on experience.

[2144] Input: Overall diagnostic results

[2145] Output: Display diagnostic results to the user

[2146] Step 8:

[2147] Providing a virtual try-on experience

[2148] The smart glasses provide users with a virtual try-on experience based on the diagnostic results received from the server, allowing them to try on clothes in a virtual space without actually going to a store.

[2149] Input: Diagnosis results, face and body images from smart glasses

[2150] Output: Virtual try-on experience

[2151] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[2152] MODE FOR CARRYING OUT THE INVENTION

[2153] System program and processing description

[2154] The system of this invention utilizes AI and an emotion engine to help users find fashion and makeup items that suit them. The system operates in cooperation with a server, a terminal, and a user, each of which plays a specific role.

[2155] Server Operation

[2156] 1. Receiving profile information:

[2157] The server receives the user's profile information (age, gender, fashion preferences, etc.) sent from the device and stores it in a database.

[2158] 2. Receiving and analyzing face images:

[2159] The server receives the user's facial image from the device, applies an image analysis algorithm to analyze the user's bone structure and personal color, and temporarily stores the facial image analysis results.

[2160] 3. How the Emotion Engine works:

[2161] The server analyzes the user's emotions in real time based on the received facial images and dialogue content, and recognizes their emotional state, which is used in the next processing step.

[2162] 4. Interactive question generation and answer analysis:

[2163] The server generates interactive questions based on the emotion engine, taking into account the user's momentary emotional state, and asks the questions to the user via the device. It receives the user's answers, analyzes their content, and dynamically generates the next questions to ask. This process is repeated until all necessary information is obtained.

[2164] 5. Generating comprehensive diagnostic results:

[2165] The server integrates the received profile information, facial image analysis results, emotional state, and user responses, and uses an AI model to generate a comprehensive diagnostic result, which includes suitable fashion items, brands, and shops for the user.

[2166] 6. Presentation of results and recommendations:

[2167] Based on the generated diagnostic results, the server generates suggestions for specific fashion items, brands, and shops that take into account the emotional state, and transmits them to the terminal to present to the user.

[2168] Device behavior

[2169] 1. Provide profile information input interface:

[2170] The terminal provides an interface for the user to input profile information and transmits the input information to the server.

[2171] 2. Provide face image upload interface:

[2172] The terminal provides an interface for the user to upload a facial image and transmits the uploaded facial image to the server.

[2173] 3. Providing an interactive question and answer input interface:

[2174] The terminal displays the interactive questions sent from the server, provides an interface for the user to answer the questions, and sends the user's answers to the server.

[2175] 4. Provide diagnostic result display interface:

[2176] The terminal provides an interface for displaying the diagnosis results and suggestions sent from the server to the user.

[2177] User behavior

[2178] 1. Enter your profile information:

[2179] Through the device interface, users enter their profile information, including age, gender, and fashion preferences.

[2180] 2. Upload your face image:

[2181] Users upload their facial images using the device's interface.

[2182] 3. Answer the interactive questions:

[2183] The user answers questions sent from the server through the device. For example, to the question "What color clothes do you like?", the user answers "I like pastel colors."

[2184] 4. Review the diagnostic results and accept the recommendations:

[2185] Users can check the diagnostic results displayed on their device and browse suggested fashion items, brands, and shops.

[2186] Specific examples

[2187] Example 1: A woman in her 30s who likes casual fashion

[2188] 1. User: Accesses the app and enters age (30s), gender (female), and fashion preference (casual).

[2189] 2. Terminal: Sends input information to the server.

[2190] 3. User: Upload a photo of your face.

[2191] 4. Device: Sends a photo of your face to the server.

[2192] 5. Server: Performs facial image analysis and determines the person as a "spring type." The analysis results are temporarily saved.

[2193] 6. Server: Uses an emotion engine to recognize emotions from the user's facial images and answers, and understands their current emotional state.

[2194] 7. Server: Generates questions and asks the user, "What color clothes do you like?". Adjusts the questions depending on the user's emotional state.

[2195] 8. Terminal: displays the question and accepts the user's answer.

[2196] 9. User: "I like pastel colors."

[2197] 10. Terminal: Sends the answer to the server.

[2198] 11. Server: Analyzes the answer and generates the next question. Sends "What kind of clothes would you like to wear for what occasion?". Adjusts the next question based on the emotional state.

[2199] 12. Terminal: Show question.

[2200] 13. User: Enter "Relaxing holiday scene."

[2201] 14. Terminal: Sends the answer to the server, analyzes the answer, and repeats this process until all the necessary information is gathered.

[2202] 15. Server: Using an AI model based on facial image analysis results, answers to interactive questions, and emotional state, the server generates diagnostic results suitable for casual fashion, including information on "pastel-colored items" and "suitable brands."

[2203] 16. Server: Dynamically generates diagnostic results and specific recommendations.

[2204] 17. Terminal: Displays the diagnostic results and suggestions to the user.

[2205] 18. User: Based on the presented results, select a pastel-colored casual fashion item and purchase it from the suggested shop.

[2206] In this way, by combining emotion engines, it is possible to provide highly accurate personalized fashion advice according to the user's emotional state.

[2207] The processing flow will be explained below.

[2208] Program processing steps

[2209] Step 1: Enter your user information

[2210] User: Visits an app or website and enters profile information such as age, gender, and fashion preferences.

[2211] Terminal: Receives profile information entered by the user and sends it to the server.

[2212] Server: Stores the received profile information in a database and prepares for the next step.

[2213] Step 2: Upload and analyze face images

[2214] Users: Upload a photo of themselves to an app or website.

[2215] Terminal: Sends the uploaded face photo to the server.

[2216] Server: Analyzes bone structure and personal color using facial image analysis algorithms. Saves the analysis results temporarily and uses them in the next step.

[2217] Step 3: Emotion analysis using the emotion engine

[2218] Server: Analyzes the user's emotions in real time using an emotion engine based on facial images and dialogue content. Temporarily stores the analysis results.

[2219] Step 4: Interactive question generation and answer analysis

[2220] Server: Generates conversational questions that take into account the user's emotional state and sends them to the device. For example, set questions such as "What color clothes do you like?" or "What kind of clothes do you like to wear for what occasions?"

[2221] Terminal: Presents an interface for displaying interactive questions to the user and entering answers.

[2222] User: Enters an answer to a question.

[2223] Terminal: Sends the entered answer to the server.

[2224] Server: Analyzes the received answers and dynamically generates the next question to ask. This process is repeated until all necessary questions have been asked.

[2225] Step 5: Generate comprehensive diagnostic results

[2226] Server: Integrates the results of facial image analysis, answers collected through dialogue questions, and emotional state, and uses AI models to generate comprehensive diagnostic results. This identifies the user's personal color, clothing styles and designs that suit them, popular brands, and suitable accessories and makeup items.

[2227] Server: Stores the diagnostic results in a database and prepares the results for display.

[2228] Step 6: Present your results and make recommendations

[2229] Server: Based on the diagnosis results, it generates recommendations for specific fashion items, brands, and shops and sends them to the device. The recommendations may be updated in real time.

[2230] Terminal: Provides an interface for displaying the diagnostic results and suggestions sent from the server to the user.

[2231] User: Based on the results presented, users can choose fashion and makeup items that suit them best.

[2232] Specific examples

[2233] Example: A woman in her 30s who likes casual fashion

[2234] 1. Step 1:

[2235] User: Accesses the app and enters age (30s), gender (female), and fashion preference (casual).

[2236] Terminal: Sends input information to the server.

[2237] Server: Saves the input information in a database.

[2238] 2. Step 2:

[2239] User: Upload a photo of themselves to the app.

[2240] Device: Sends a photo of your face to the server.

[2241] Server: Performs facial image analysis and determines "spring type." Temporarily saves the analysis results.

[2242] 3. Step 3:

[2243] Server: Uses an emotion engine to recognize emotions from the user's facial image and responses, and understands their current emotional state. For example, it determines whether the user is in a "peace of mind" state.

[2244] 4. Step 4:

[2245] Server: Based on the emotional state, generate a question like "What color clothes do you like?" and send it to the user. The question content is adjusted depending on the emotional state.

[2246] Terminal: Provides an interface for displaying questions and accepting user answers.

[2247] User: "I like pastel colors."

[2248] Terminal: Sends the answer to the server.

[2249] Server: Analyzes the answer and generates the next question: "What kind of occasions do you want to wear your clothes for?" The next question is adjusted taking into account the emotional state.

[2250] Terminal: Display the question.

[2251] User: Enter "Relaxing holiday scene."

[2252] Device: Sends the answer to the server and repeats this process until it has gathered the required information.

[2253] 5. Step 5:

[2254] Server: Using an AI model based on facial image analysis, responses to interactive questions, and emotional state, the server generates a comprehensive diagnosis suitable for casual fashion, including information on "pastel-colored items" and "suitable brands."

[2255] Server: Stores the diagnostic results in a database.

[2256] 6. Step 6:

[2257] Server: Generates diagnostic results and specific fashion item, brand, and shop recommendations based on the results. Suggestions may be updated in real time.

[2258] Terminal: Provides an interface for displaying diagnostic results and recommendations to the user.

[2259] User: Based on the presented results, select a pastel-colored casual fashion item and purchase it from the suggested shop.

[2260] In this way, users can receive highly accurate and realistic fashion advice that takes their emotions into consideration.

[2261] Example 2

[2262] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[2263] Conventional fashion advice systems often make suggestions based solely on the user's profile information or image analysis results, and are unable to consider the user's emotional state or real-time changes in preferences. This makes it difficult to provide personalized, highly accurate fashion advice, resulting in reduced user satisfaction. The present invention aims to solve these problems and provide personalized fashion advice that takes the user's emotional state into account.

[2264] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[2265] In this invention, the server includes a means for receiving input information from a user, a means for receiving a facial image of the user and analyzing facial features and color characteristics through image processing, and a means for analyzing the user's emotions in real time and dynamically generating questions based on the analysis results. This makes it possible to integrate the user's profile information, facial image analysis results, emotion analysis results, and user answers, use a generative AI model to generate highly accurate comprehensive diagnostic results, and suggest clothing items, brands, and sales locations that are suitable for the user.

[2266] "Input Information" means the personal information and profile data that a User provides to the System.

[2267] A "face image" refers to a photograph or image data of a user's face.

[2268] "Image processing" refers to the technology of analyzing image data and extracting specific features or attributes.

[2269] "Facial features" refers to the shape, bone structure, and feature points of the user's face.

[2270] "Color characteristics" refers to a personal color classification based on the user's face and skin color.

[2271] "Emotion analysis" is a technology that analyzes a user's emotional state in real time from facial images and dialogue content.

[2272] A "generative AI model" is an artificial intelligence model that uses machine learning and natural language processing to analyze data and generate interactive questions and diagnostic results.

[2273] "Comprehensive diagnosis results" are diagnostic results generated by the AI ​​model based on the user's profile information, facial image analysis results, emotion analysis results, and the user's responses.

[2274] "Clothing items" refers to clothes, accessories, etc. suggested to users.

[2275] "Trademark" means the name or logo of a particular brand or manufacturer, which is associated with the proposed fashion item.

[2276] "Point of Sale" refers to a store or online shop where the products suggested to the user can be purchased.

[2277] This invention is a system that utilizes AI and an emotion engine to help users find fashion and makeup items that suit them. The system works in cooperation with a server, a device, and a user, each of which plays a specific role.

[2278] Server Operation

[2279] 1. Receiving profile information

[2280] The server receives the user's input information (age, gender, fashion preferences, etc.) sent from the device and stores it in a database. Specifically, it uses a MySQL database.

[2281] 2. Receiving and analyzing face images

[2282] After receiving the user's facial image from the device, the server processes the image using OpenCV and Dlib. The facial image is analyzed to analyze the user's skeletal and color characteristics, and the analysis results are temporarily stored in Redis.

[2283] 3. Operation of the Emotion Engine

[2284] The server uses Microsoft Azure's emotion analysis API to analyze the user's emotions in real time from facial images and dialogue content, recognizing their emotional state. The analysis results are used in the next step of question generation.

[2285] 4. Question Generation and Answer Analysis

[2286] The server generates questions that take into account the user's temporary emotional state based on the results of sentiment analysis. The generated questions are sent to the user via the device and the user's answers are received. The received answers are then analyzed and the next question is dynamically generated. The NLP technology used is SpaCy and the BERT model.

[2287] 5. Generating comprehensive diagnostic results

[2288] The server integrates the received input information, facial image analysis results, emotional state, and user responses, and uses a generative AI model (e.g., GPT-4) to generate a comprehensive diagnosis result, which includes clothing items, brands, and sales locations suitable for the user.

[2289] 6. Presentation of results and recommendations

[2290] Based on the generated diagnostic results, the server generates specific suggestions that take into account the emotional state, and transmits them to the terminal to present to the user.

[2291] Device behavior

[2292] 1. Provides a profile information input interface

[2293] The terminal provides an interface for users to input information and sends the input information to the server, specifically using JavaScript and ReactJS.

[2294] 2. Providing a face image upload interface

[2295] The terminal provides an interface for the user to upload a facial image and transmits the uploaded facial image to the server.

[2296] 3. Providing a question and answer input interface

[2297] The terminal provides an interface for displaying the questions sent from the server and accepting the user's answers, and then sends the user's answers to the server.

[2298] 4. Providing an interface for displaying diagnostic results

[2299] The terminal provides an interface for displaying the diagnosis results and suggestions sent from the server to the user.

[2300] User behavior

[2301] 1. Enter your profile information

[2302] Through the device's interface, users enter their personal information, including age, gender, and fashion preferences.

[2303] 2. Upload a face image

[2304] Users upload their facial images using the device's interface.

[2305] 3. Answering questions

[2306] The user answers questions sent from the server through the device. For example, to the question "What color clothes do you like?", the user answers "I like pastel colors."

[2307] 4. Review the diagnostic results and accept the recommendations

[2308] The user checks the diagnostic results displayed on the device and browses suggested clothing items, brands, and sales locations.

[2309] Specific examples

[2310] Example 1: A woman in her 30s who likes casual fashion

[2311] 1. User: Accesses the app and enters age (30s), gender (female), and fashion preference (casual).

[2312] 2. Terminal: Sends input information to the server.

[2313] 3. Server: Stores profile information in a MySQL database.

[2314] 4. User: Upload a photo of your face.

[2315] 5. Device: Sends a photo of your face to the server.

[2316] 6. Server: Performs facial image analysis and determines "spring type." Temporarily stores the analysis results in Redis.

[2317] 7. Server: Uses an emotion engine to recognize emotions from the user's facial images and answers, and understands their current emotional state.

[2318] 8. Server: Generates questions and asks the user, "What color clothes do you like?". Adjusts the questions depending on the user's emotional state.

[2319] 9. Terminal: displays the question and accepts the user's answer.

[2320] 10. User: "I like pastel colors."

[2321] 11. Terminal: Sends the answer to the server.

[2322] 12. Server: Analyzes the answer and generates the next question. Sends "What kind of clothes would you like to wear for what occasion?". Adjusts the next question based on the emotional state.

[2323] 13. Terminal: Display the question.

[2324] 14. User: Enter "Relaxing holiday scene."

[2325] 15. Terminal: Sends the answer to the server, analyzes the answer, and repeats this process until all the necessary information is gathered.

[2326] 16. Server: Using a generative AI model based on facial image analysis results, answers to interactive questions, and emotional state, the server generates diagnostic results suitable for casual fashion, including information on "pastel-colored items" and "suitable brands."

[2327] 17. Server: Dynamically generates diagnostic results and specific recommendations.

[2328] 18. Terminal: Display the diagnostic results and suggestions to the user.

[2329] 19. User: Based on the presented results, select a pastel-colored casual fashion item and purchase it at the suggested retail location.

[2330] Prompt Sentence Examples

[2331] "I'm a woman in my 30s who likes casual fashion. My face image indicates I'm a spring type. What fashion items and brands would you suggest?"

[2332] In this way, it is possible to provide personalized fashion advice according to the user's emotional state.

[2333] The flow of the identification process in the second embodiment will be described with reference to FIG.

[2334] Step 1:

[2335] Enter profile information

[2336] The user enters their own information (age, gender, fashion preferences, etc.) through the device interface. The device validates the entered information using JavaScript and sends it to the server. The server receives the input information and saves it in a MySQL database using INSERT statements. The input is the user's profile information, and the output is the profile information saved in the database.

[2337] Step 2:

[2338] Upload a face image

[2339] A user uploads their facial image using the device interface. The device encodes the uploaded facial image into Base64 format and sends it to the server. The server receives the facial image and temporarily stores it in a file system or database. The input is the user's facial image, and the output is the facial image stored on the server.

[2340] Step 3:

[2341] Facial image analysis

[2342] The server analyzes the stored facial images using OpenCV and Dlib. First, it uses a face detection algorithm to extract facial feature points, then analyzes the bone structure and color characteristics. The analysis results are temporarily stored in Redis. The input is the user's facial image, and the output is the analysis results including bone structure and color characteristics.

[2343] Step 4:

[2344] Emotion analysi...

Claims

1. means for receiving profile information from a user; A means for receiving a facial image of a user and analyzing bone structure and personal color by image analysis; A means of interactively asking users questions and analyzing their responses; a means for integrating the received profile information, facial image analysis results, and user responses to generate a comprehensive diagnostic result using an AI model; and A means for suggesting fashion items, brands, and shops suitable for the user based on the diagnosis results; A system including:

2. 10. The system of claim 1, wherein the system dynamically changes user preferences and characteristics based on real-time analysis.

3. 10. The system of claim 1, wherein the system dynamically generates the next question based on the user's answer.

Images