System

Abstract

A system is provided.SOLUTION: A system, comprising: means for enabling a user to upload a picture of his / her face and skeleton; means for a server to receive the uploaded picture and extract features of the face and skeleton; means for matching the extracted features with a database of fashion models and calculating an optimal fashion for the user; means for generating and transmitting advice on optimal fashion items and styles to a user terminal; and means for displaying the generated advice on a user interface.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] It is a difficult task for users to find the fashion that best suits their body shape and facial features. It is particularly difficult for women in their teens to thirties to accurately determine which style suits them. The problem that this invention aims to solve is to provide statistically optimal fashion styles based on the user's own facial and skeletal features, allowing the user to dress with confidence. [Means for solving the problem]

[0005] The system of the present invention includes means for allowing users to upload photographs of their face and bone structure, means for a server to receive the uploaded photographs and extract facial and bone structure features, means for comparing the extracted features with a fashion model database to calculate the most suitable fashion for the user, means for generating advice on the most suitable fashion items and style and sending it to the user terminal, and means for displaying the generated advice on a user interface. This system allows users to easily find the most suitable fashion style based on their own features.

[0006] "User" refers to a person who uses the system to upload photos of their face and bone structure and receive fashion advice.

[0007] "Photograph" refers to image data that captures the user's facial and skeletal features.

[0008] "Upload" refers to the act of a user sending photo data from their device to a server.

[0009] "Server" refers to a computer system that receives photos uploaded by users and processes the data.

[0010] "Facial and skeletal features" refer to the individual characteristics that describe the shape of a user's face and the structure of their body.

[0011] A "fashion model database" refers to data storage that accumulates data on fashion models with various facial shapes and body shapes.

[0012] "Matching" refers to the process of comparing a user's facial and skeletal features with a database of fashion models to find a matching model.

[0013] "Fashion items" refer to specific items related to fashion, such as clothing, accessories, and shoes.

[0014] "Style" refers to the way certain fashion items are combined and worn.

[0015] "Advice" refers to specific suggestions regarding fashion items and styles that are best suited to the user based on their characteristics.

[0016] "User interface" refers to the screen layout and operation method that allows a user to perform operations and check information through an application. [Brief explanation of the drawings]

[0017] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 3 is a sequence diagram showing a processing flow of the data processing system according to the first embodiment. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0018] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0019] First, the terms used in the following description will be explained.

[0020] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0021] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0022] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0023] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0024] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0025] [First embodiment]

[0026] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0027] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0028] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0029] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0030] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0031] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0032] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0033] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0034] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0035] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0036] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0037] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0038] Overall system configuration

[0039] This invention is a system that allows users to upload photos of their face and bone structure and provides fashion advice based on those photos. The system is composed of a user terminal, a server, and a database of fashion models.

[0040] User device functions

[0041] The user device has an application that allows users to take and select photos and upload them to the server, and also has the function of displaying fashion advice sent from the server.

[0042] Example: A user launches an app, uses the camera to take a photo of their face and body, and then uploads the photo to a server through the app's interface.

[0043] Server Features

[0044] The server receives photos uploaded from the user's device and analyzes facial and skeletal features using image processing technology and machine learning algorithms. After analysis, the photos are compared with a database of fashion models to calculate the most suitable fashion items and style for the user. Final advice is then sent to the user's device.

[0045] Example: The server receives the uploaded photo and applies a facial recognition algorithm to extract facial shape and skeletal features. The extracted data is compared with a database to select the most suitable fashion model. For example, if the user's characteristics are "long face, slim figure," the server will search the database for a model with the closest characteristics.

[0046] Fashion Model Database Features

[0047] This database stores information on fashion models with various facial shapes and bone structures. The database includes each model's facial shape, body shape, recommended fashion style, and item list.

[0048] Example: If a model with a "round face and plump figure" is registered in the database, the style and items recommended by that model (e.g., A-line dresses, flared pants, etc.) will be recorded.

[0049] Fashion advice generation and transmission to user devices

[0050] After the server selects the optimal model, it generates specific fashion advice based on the model's data. The advice includes recommended fashion items and styling details. The advice is then sent to the user's device and displayed on a user interface.

[0051] Example: Based on the user's characteristics, specific styling advice is generated, such as "V-neck tops and high-waisted pants look good on you." This advice is sent to the user's device and displayed within the app.

[0052] Use Case Details

[0053] For example, if a user has a "long face and slim figure," the server extracts the corresponding features from the database and determines that "Model A" is the best fit. Model A recommends that "V-neck tops and high-waisted pants look good on you," and this styling advice is provided to the user. The user can check this advice within the app and use it as a reference for shopping and coordinating outfits.

[0054] The system allows users to receive accurate fashion advice based on their facial and bone structure, making it easy to find the style that best suits them.

[0055] The processing flow will be explained below.

[0056] Step 1:

[0057] The user launches the smartphone app and takes a photo of their face and whole body, or selects an existing photo.

[0058] Step 2:

[0059] Users upload photos to the server through the app's interface, and the photo data is transmitted over secure communication.

[0060] Step 3:

[0061] The server receives photo data uploaded by the user.

[0062] Step 4:

[0063] The server passes the received photo data to the image processing module for analysis, where the face detection and feature extraction process begins.

[0064] Step 5:

[0065] The server's image processing module extracts facial shape and skeletal features, such as facial contours, eye, nose, and mouth positions, and skeletal proportions.

[0066] Step 6:

[0067] The server then compares the extracted feature data with a database of fashion models, which contains data on models with various facial and body shapes.

[0068] Step 7:

[0069] The server searches the database for a model that best matches the user's characteristics and calculates the optimal fashion style based on that model.

[0070] Step 8:

[0071] The server generates advice on optimal fashion items and styles, including specific clothing types and styling suggestions.

[0072] Step 9:

[0073] The server transmits the generated fashion advice to the user's terminal.

[0074] Step 10:

[0075] The user's device receives the advice data sent from the server and displays it within the application, allowing the user to check it and use it to help them choose their own fashion.

[0076] Example 1

[0077] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0078] Conventional fashion advice systems often have difficulty providing personalized advice that takes into account the user's facial and body shapes. Furthermore, they have faced the problem of requiring time and effort for users to find the fashion items and styling that best suit them. This makes it difficult for users to easily find the style that best suits them.

[0079] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0080] In this invention, the server includes means for allowing users to upload photos of their face and skeletal structure, means for receiving the uploaded photos and extracting facial and skeletal features, means for comparing the extracted features with a fashion model database to calculate the most suitable fashion for the user, means for generating advice on the most suitable fashion items and style and sending it to the user terminal, means for displaying the generated advice on a user interface, and means for performing a facial recognition algorithm, a machine learning algorithm, and database comparison. This allows users to receive accurate fashion advice based on their facial and skeletal features, making it easy to find the style that best suits them.

[0081] "User Device" means a device equipped with an application that allows a user to take and select photos and upload them to the server, and that has the functionality to display fashion advice sent from the server.

[0082] The "server" is a central processing unit that receives photos uploaded from the user's terminal, analyzes facial and skeletal features, compares the analysis results with a database of fashion models to calculate the most suitable fashion items and style for the user, and sends the generated advice to the user's terminal.

[0083] A "fashion model database" is a database that stores information on fashion models with various facial shapes and skeletal features, and is a data storage that includes each model's facial shape, body shape, recommended fashion style, item list, etc.

[0084] A "facial recognition algorithm" is an algorithm used to analyze the shape and features of a face from an uploaded photo, and is a technology that uses image processing libraries such as OpenCV and Dlib to detect facial contours and feature points.

[0085] A "machine learning algorithm" is an algorithm that performs classification and prediction based on extracted facial and skeletal feature data, and is a technology that analyzes feature data using methods such as Random Forest and SVM (Support Vector Machine).

[0086] "Fashion items" are specific clothing, accessories, and other fashion products that are suggested to a user and are recommended based on the user's facial and skeletal features.

[0087] A "prompt" is an instruction entered into a generative AI model to output appropriate fashion advice, and is text input to obtain optimal output based on the user's characteristics and requests.

[0088] MODE FOR CARRYING OUT THE INVENTION

[0089] Overall system configuration

[0090] This invention is a system that allows users to upload photos of their face and bone structure and provides fashion advice based on those photos. The system is composed of a user terminal, a server, and a database of fashion models.

[0091] User device functions

[0092] The user device has an application that allows users to take and select photos and upload them to the server. It also has a function to display fashion advice sent from the server. Users launch the app, take photos of their face and body using the camera, and then upload the photos to the server through the app interface.

[0093] Examples:

[0094] The user launches the app on their smartphone, takes a photo of their face and whole body, and then clicks the app's upload button to send the photo to the server.

[0095] Server Features

[0096] The server receives photos uploaded from the user's device and analyzes facial and skeletal features. This analysis uses image processing technology and machine learning algorithms. Specifically, OpenCV and Dlib are used to detect facial shape and feature points, and the data is analyzed using machine learning algorithms such as Random Forest and SVM. After analysis, the results are compared with a fashion model database to calculate the optimal fashion items and style for the user. Final advice is then sent to the user's device.

[0097] Examples:

[0098] The server receives the photo sent by the user and performs facial recognition using OpenCV. It extracts facial feature points using Dlib and then applies the Random Forest algorithm to identify the user's facial and skeletal characteristics. It then compares the results with a fashion model database to select a model that most closely matches the "long face, slim body shape" description and calculates the model's recommended fashion style.

[0099] Fashion Model Database Features

[0100] This database stores information on fashion models with various facial shapes and bone structures. The database includes each model's facial shape, body shape, recommended fashion style, and item list.

[0101] Examples:

[0102] If a model with a "round face and plump figure" is registered in the database, the style and items that model recommends (e.g., A-line dresses, flared pants, etc.) are recorded.

[0103] Fashion advice generation and transmission to user devices

[0104] After the server selects the optimal model, it generates specific fashion advice based on the model's data. The advice includes recommended fashion items and styling details. The advice is sent to the user's device and displayed on a user interface.

[0105] Examples:

[0106] Based on the user's characteristics, specific styling advice is generated, such as "V-neck tops and high-waisted pants look good on you." This advice is sent to the user's device and displayed within the app.

[0107] Prompt Sentence Examples

[0108] A typical example of a prompt to be input to a generative AI model might be:

[0109] "Based on photos of the user's face and body, please suggest the most suitable fashion items and styles. The user's characteristics are as follows: 'Long face, slim body'. Please output the most suitable fashion advice for this."

[0110] Using this prompt, the generative AI model outputs content that provides appropriate fashion advice.

[0111] The system allows users to receive accurate fashion advice based on their facial and bone structure, making it easy to find the style that best suits them.

[0112] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0113] Step 1:

[0114] Receiving and storing photos on the server

[0115] The server receives photos uploaded from the user's device. This input is a face and full-body photo sent by the user through the app. The server saves the received photos in temporary storage. As a concrete example, the server receives image data through an HTTP request and saves the data in local storage.

[0116] Input: Face and full-body photo data sent from the user device.

[0117] Data processing: Analyzing HTTP requests and saving image data.

[0118] Output: Saved image file.

[0119] Step 2:

[0120] Server-based face recognition and skeletal feature extraction

[0121] The server analyzes the stored photos to extract facial shape and skeletal features. This step uses image processing technologies such as OpenCV and Dlib. First, a facial recognition algorithm is applied, and then Dlib is used to extract the coordinates of facial feature points (eyes, nose, mouth, etc.).

[0122] Input: Saved face and full-body image files.

[0123] Data processing: Face recognition using OpenCV and feature point extraction using Dlib.

[0124] Output: Facial shape and skeletal feature coordinate data.

[0125] Step 3:

[0126] Application of machine learning algorithms by the server

[0127] The server inputs the extracted facial and skeletal feature data into a machine learning algorithm to classify the user's face shape and body shape. The algorithms used include Random Forest and SVM. In this step, the feature data is input into the machine learning model as a numerical vector to obtain the classification results.

[0128] Input: Coordinate data of facial shape and skeletal features.

[0129] Data processing: Vectorizing feature data and inputting it into a machine learning model.

[0130] Output: Classified face shape and body shape data.

[0131] Step 4:

[0132] Database verification by the server

[0133] The server compares the classified face shape and body shape data with a fashion model database, which contains each model's face shape, body shape, recommended fashion style, and items. In this step, the server searches for the most suitable model and obtains the model's data.

[0134] Input: Classified face shape and body shape data.

[0135] Data processing: Search the fashion model database and select the most suitable model.

[0136] Output: Data of the best fashion model.

[0137] Step 5:

[0138] Server-based fashion advice generation

[0139] The server generates specific fashion advice based on the data of the selected fashion model. Using the generative AI model, it generates text that suggests the most suitable fashion items and styles for the user. For example, it might generate advice such as "V-neck tops and high-waisted pants look good on you."

[0140] Input: Data of the optimal fashion model.

[0141] Data processing: Generate text advice using a generative AI model.

[0142] Output: Text data of fashion advice.

[0143] Step 6:

[0144] Server sends advice and displays it on the user's device

[0145] The server sends the generated fashion advice to the user's device, which then displays the received advice within the application. Specifically, the server sends the advice as an HTTP response, and the user's device analyzes it and displays it on the interface.

[0146] Input: Text data of fashion advice.

[0147] Data processing: Sending advice and analyzing received data.

[0148] Output: User interface with advice displayed.

[0149] (Application example 1)

[0150] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0151] Fashion-conscious users face challenges in easily finding the styles and items that best suit them. In particular, there are few methods for quickly providing personalized fashion advice based on a user's face shape and bone structure, making it time-consuming and labor-intensive to select a style that suits them. Furthermore, there is no way to immediately purchase products based on the advice provided, which can disrupt the user's shopping experience. To address these issues, a system is needed that can quickly and accurately analyze a user's characteristics, provide optimal fashion advice, and enable the user to purchase products based on the advice.

[0152] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0153] In this invention, the server includes means for allowing a user to upload images of their face and bone structure, means for receiving the uploaded images and extracting facial and bone structure features, means for comparing the extracted features with a fashion model database to calculate the most suitable fashion for the user, means for generating advice on the most suitable fashion items and style and transmitting it to the user terminal, means for displaying the generated advice on a user interface, and means for allowing the user to purchase products based on the advice through an application. This allows the user to receive the most suitable fashion advice based on their own features and instantly purchase products based on the advice.

[0154] A "user terminal" is a device that allows a user to take photos, upload images to a server through an application, and receive fashion advice.

[0155] The "server" is a computer system that receives images sent from a user terminal, analyzes them, generates fashion advice, and sends it to the user terminal.

[0156] "Image processing technology" is a technology for analyzing images taken by a user and extracting facial and skeletal features.

[0157] "Machine learning algorithms" are artificial intelligence technologies that compare facial and skeletal features with a database to generate optimal fashion advice for users.

[0158] A "fashion model database" is a collection of information that stores data on models based on various face shapes and body shapes, and is used to provide fashion advice based on that data.

[0159] The "user interface" refers to a screen or operating means for displaying the generated fashion advice to the user.

[0160] A "prompt sentence" is text data that uses a generative AI model to interactively analyze the user's image and features.

[0161] The "shopping function" is a function that allows a user to directly purchase items shown in the advice based on the generated fashion advice.

[0162] This invention is a system that allows users to upload images of their face and bone structure and provides fashion advice based on those images. The system consists of a user terminal, a server, and a database of fashion models.

[0163] User device functions

[0164] The user terminal has a function that allows the user to take pictures and upload them to the server through the application, and also provides an interface that displays fashion advice sent from the server and allows the user to purchase products based on that advice.

[0165] For example, a user launches the app and takes a picture of their face and whole body using their camera. They then upload the image to a server through the app's interface. After uploading, the user waits for a response from the server and can view the resulting advice on the screen. Based on this advice, a link or button to purchase the product is provided, allowing the user to continue shopping.

[0166] Server Features

[0167] The server receives the images sent from the user terminal and extracts facial and skeletal features using image processing techniques and machine learning algorithms.

[0168] Specifically, the server does the following:

[0169] 1. Image reception: Has an API endpoint for receiving image files sent from the user device.

[0170] 2. Feature extraction: The received image is analyzed and facial and skeletal features are extracted using a facial recognition algorithm, such as the Python face_recognition library. Numerical data such as facial shape and skeletal proportions are obtained.

[0171] 3. Database Matching: The extracted features are matched against a database of fashion models, which contains feature data for each model and fashion styles based on those features.

[0172] 4. Advice Generation: Select the best model and generate styling advice based on that model. This advice is crafted in detail using the generative AI model and prompt text.

[0173] 5. Sending advice: Install an API to send the generated advice to the user's device.

[0174] Fashion Model Database Features

[0175] The fashion model database stores information on models with various facial shapes and bone structures. The database includes each model's facial shape, body shape, recommended fashion style, and item list.

[0176] For example, if a model with a "round face and plump figure" is registered in the database, the style and clothing that model recommends (e.g., A-line dresses, flared pants, etc.) are recorded. This allows the server to provide advice that best suits the user's characteristics.

[0177] Fashion advice generation and transmission to user devices

[0178] After the server selects the optimal model, it generates specific fashion advice based on the model's data. The advice includes recommended fashion items and styling details. The advice is sent to the user's device and displayed on a user interface.

[0179] For example, styling advice such as "V-neck tops and high-waisted pants look good on you" is generated based on the user's characteristics. This advice is sent to the user's device and displayed within the app. The user can also purchase items directly within the app.

[0180] Prompt Sentence Examples

[0181] "Analyze the user's photo and extract facial shape and bone structure features. Based on those features, provide the most suitable fashion items and styling advice."

[0182] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0183] Step 1:

[0184] The user launches the smartphone app and takes a photo of their face and whole body.

[0185] In this step, the user takes a photo using the app's camera function and the photo is saved in the smartphone. The input is the original photo taken by the user, and the output is the photo data saved in the smartphone.

[0186] Step 2:

[0187] The user terminal uploads the photograph to the server.

[0188] In this step, when a user clicks the upload button in the app, the photo file is sent to the server through the HTTP request process within the app. The input is the photo data stored on the smartphone, and the output is the data sent to the server.

[0189] Step 3:

[0190] The server receives the photo data and uses image processing techniques and machine learning algorithms to extract facial and skeletal features.

[0191] Specifically, the server analyzes the photo using Python's face_recognition library and extracts the facial shape and skeletal features as numerical data. The input is the photo data sent to the server, and the output is the extracted facial and skeletal feature data.

[0192] Step 4:

[0193] Based on the extracted feature data, the server compares it with a database of fashion models and calculates the most suitable fashion for the user.

[0194] The server uses the extracted feature data to compare it with the features of each model stored in the database and selects the optimal model. The input is the extracted feature data and a database of fashion models, and the output is the optimal model data and styling advice.

[0195] Step 5:

[0196] The server uses the generated AI model and prompts to generate specific fashion advice.

[0197] In this step, the server inputs prompts into the generative AI model to generate specific fashion item and style advice based on the user's characteristics. The input is the optimal model's data and prompts, and the output is specific fashion advice.

[0198] Step 6:

[0199] The server transmits the generated advice to the user terminal.

[0200] The server sends the generated advice to the user terminal as an HTTP response. The input is the specific fashion advice, and the output is the advice data sent to the user terminal.

[0201] Step 7:

[0202] The user terminal displays the received advice on a user interface, and the user purchases the product based on the advice.

[0203] In this step, the user can review the advice displayed within the app and purchase the product by clicking a purchase link or button. The input is the advice data received from the server, and the output is the advice and purchase procedure displayed on the user's display screen.

[0204] Prompt Sentence Examples

[0205] "Analyze the user's photo and extract facial shape and bone structure features. Based on those features, provide the most suitable fashion items and styling advice."

[0206] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0207] Overall system configuration

[0208] This invention is a system that allows users to upload photos of their face and bone structure, and provides fashion advice based on the user's emotions. The system consists of a user terminal, a server, an emotion engine, and a database of fashion models.

[0209] User device functions

[0210] The user device is equipped with an application that allows users to take and select photos and upload them to the server, and also has the function of displaying fashion advice and personalized recommendation advice based on emotions sent from the server.

[0211] Example: A user launches an app, uses the camera to take a photo of their face and body, captures facial expressions to recognize their current emotion, and then uploads the photo to a server through the app interface.

[0212] Server Features

[0213] The server receives photos and facial expression data uploaded from the user's device and analyzes facial and skeletal features and emotions. The analysis uses image processing technology, machine learning algorithms, and an emotion engine. After analysis, the server compares the results with a database of fashion models and calculates advice based on the user's optimal fashion items, style, and emotions. The final advice is then sent to the user's device.

[0214] Example: The server receives the uploaded photo and facial expression data, and applies facial recognition and emotion recognition algorithms to extract facial shape, skeletal features, and the current emotional state (e.g., "joy" or "sadness"). The extracted data is compared with a database to select the most suitable fashion model.

[0215] Emotion Engine Functions

[0216] The emotion engine is a module for recognizing emotions from facial expression data uploaded by users. This engine uses machine learning algorithms to analyze the user's facial expressions and identify the emotion at that time.

[0217] Example: If the emotion engine recognizes the emotion "joy" from a user's facial expression, it will recommend fashion items that match that emotion, including relaxed styles and cheerful color palettes.

[0218] Fashion Model Database Features

[0219] This database stores information on fashion models with various facial shapes and bone structures. The database includes each model's facial shape, body shape, recommended fashion style, and item list.

[0220] Example: If a model with a "round face and plump figure" is registered in the database, the style and items recommended by that model (e.g., A-line dresses, flared pants, etc.) will be recorded.

[0221] Fashion advice generation and transmission to user devices

[0222] After the server selects the optimal model, it generates specific fashion advice based on the model's data and the user's emotions. The advice includes recommended fashion items, styling details, and emotional suggestions. The generated advice is sent to the user's device and displayed on a user interface.

[0223] Example: Based on the user's characteristics and emotions, specific styling advice such as "V-neck tops and high-waisted pants look good on you" is provided, along with additional emotional advice such as "Bright colors are recommended for today's mood." This advice is sent to the user's device and displayed within the app.

[0224] Use Case Details

[0225] For example, if a user has a "long face, slim figure," and is currently feeling "happy," the server will extract the corresponding features and emotions from the database and determine that "Model A" is the best fit. Model A recommends that "V-neck tops and high-waisted pants look good on you," so in addition to this styling advice, it will provide the user with emotion-based advice such as "bright-colored items are perfect for your mood today." Users can check these advice within the app and use them as reference for shopping and outfits.

[0226] The system allows users to receive accurate fashion advice based on their facial and skeletal features and emotions, making it easy to find the style that best suits them.

[0227] The processing flow will be explained below.

[0228] Step 1:

[0229] The user launches the smartphone app and takes a photo of their face, their whole body, and their facial expression, or selects an existing photo.

[0230] Step 2:

[0231] Users upload photo data and facial expression data to the server through the app interface, and the data is transmitted over secure communication.

[0232] Step 3:

[0233] The server receives the photo data and facial expression data uploaded by the user.

[0234] Step 4:

[0235] The server then passes the received photo data to the image processing module for analysis, where the process of recognizing facial and skeletal features and facial expressions begins.

[0236] Step 5:

[0237] The server's image processing module extracts facial shape and skeletal features, such as facial contours, eye, nose, and mouth positions, and skeletal proportions.

[0238] Step 6:

[0239] Furthermore, an emotion engine analyzes the facial expression data to identify the user's current emotion, such as "happiness," "sadness," or "surprise."

[0240] Step 7:

[0241] The server compares the extracted feature data and emotion data with a database of fashion models, which contains data on models with various facial and body shapes.

[0242] Step 8:

[0243] The server searches the database for a model that best matches the user's facial and skeletal features and emotional data, and calculates the optimal fashion style based on that model.

[0244] Step 9:

[0245] The server generates advice about the calculated fashion items and styles, including recommended clothing types, styling details, and additional suggestions depending on the emotion.

[0246] Step 10:

[0247] The server sends the generated fashion advice to the user's device, including colors and styling that match the user's emotions.

[0248] Step 11:

[0249] The user's device receives the advice data sent from the server and displays it within the application, allowing the user to check it and use it to help them choose their own fashion.

[0250] Step 12:

[0251] Based on the advice provided, the user purchases the recommended fashion items or puts together an outfit.

[0252] Example 2

[0253] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0254] Conventional fashion advice systems only consider the user's facial and skeletal features, making it difficult to provide advice that reflects the user's current emotions. Furthermore, advice that does not consider emotions makes it difficult to select the optimal fashion that matches the user's feelings. Therefore, there is a need for a method to support users in selecting fashion that suits their daily mood and the occasion.

[0255] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0256] In this invention, the server includes a means for allowing a user to upload a photograph of their face and skeletal structure, a means for receiving the uploaded photograph and extracting facial and skeletal features, and a means for analyzing the user's emotions using an emotion engine, thereby enabling optimal fashion advice to be provided to the user based on the extracted features and analyzed emotions.

[0257] "User" refers to any person who uses the system to upload their own photos and receive fashion advice.

[0258] "Server" refers to a computer system that receives data uploaded by users and performs analysis and advice generation.

[0259] "Photo" refers to image data including a user's face and bone structure that is uploaded by the user.

[0260] "Facial and skeletal features" refers to data being analyzed that indicates the user's physical characteristics, such as facial shape and skeletal structure.

[0261] "Emotion engine" refers to a module that analyzes and identifies emotions from a user's facial expression data.

[0262] A "fashion model database" refers to an information repository that stores data on each model based on their face shape and body type.

[0263] "Means for generating advice" refers to the process of creating recommendations for the most suitable fashion items and styles for the user based on the analyzed features and emotional data.

[0264] "User Terminal" refers to the device used by a User to upload photos and receive and view advice.

[0265] "User interface" refers to the screen and interaction mechanism that displays advice generated on the user's terminal and allows the user to operate it.

[0266] "Image processing techniques" refers to techniques used to extract facial and skeletal features from photographs.

[0267] "Machine learning algorithms" refer to computational techniques used for data analysis and emotion recognition.

[0268] The system of this invention allows users to upload photos of their face and bone structure, and provides appropriate fashion advice based on the user's emotions. The system is composed of a user terminal, a server, an emotion engine, and a database of fashion models.

[0269] User device functions

[0270] The user device is equipped with an application that allows users to take and select photos and upload them to the server, and also has the function of displaying fashion advice and personalized recommendation advice based on emotions sent from the server.

[0271] Specifically, the user launches the app and takes a photo of their face and whole body using the camera. Furthermore, facial expressions are captured to recognize the current emotion, and the captured images are uploaded to a server through the app's interface. The hardware used is a mobile device such as a smartphone or tablet, and the app is developed based on a general-purpose application such as a "movie viewer for mobile devices."

[0272] Server Features

[0273] The server receives photos and facial expression data uploaded from the user's device and analyzes facial and skeletal features and emotions. Image processing technology, image processing libraries such as "OpenCV," and machine learning algorithms such as "DeepFace" are used for the analysis. The server then compares the analysis results with a database of fashion models to calculate advice based on the user's optimal fashion items, style, and emotions. The final advice is then sent to the user's device.

[0274] Specifically, the server receives a photo and facial expression data from the user, extracts facial shape and skeletal features, and simultaneously recognizes the user's emotional state (e.g., "joy" or "sadness"). The extracted data is compared with a database of fashion models, and suggestions for optimal fashion items, styling details, and emotions are generated. This data is then sent to the user's device, where the user can view it within the app.

[0275] Emotion Engine Functions

[0276] The emotion engine is a module for recognizing emotions from facial expression data uploaded by users. This engine uses machine learning algorithms to analyze the user's facial expressions and recognize specific emotions.

[0277] For example, if the emotion engine recognizes the emotion of "joy" from the user's facial expression, it will recommend fashion items that match that emotion, including a relaxed style and cheerful color palette. The machine learning algorithms used can be "TensorFlow" or "Keras."

[0278] Fashion Model Database Features

[0279] The fashion model database stores information on fashion models with various facial shapes and skeletal features. The database includes each model's facial shape, body shape, recommended fashion style, and item list.

[0280] Specifically, if a model with a "round face and plump figure" is registered in the database, the style and items recommended by that model (e.g., A-line dress, flared pants, etc.) are recorded.

[0281] Fashion advice generation and transmission to user devices

[0282] After the server selects the optimal model, it generates specific fashion advice based on the model's data and the user's emotions. The advice includes recommended fashion items, styling details, and emotional suggestions. The generated advice is sent to the user's device and displayed on a user interface.

[0283] As a specific example, based on the user's characteristics and emotions, in addition to specific styling advice such as "V-neck tops and high-waisted pants look good on you," additional emotional advice such as "bright colors are recommended for today's mood" is provided.

[0284] Prompt Sentence Examples

[0285] "You will create a program that allows users to upload photos of their face and bone structure, recognizes emotions, and provides optimal fashion advice. Using image processing technology and machine learning algorithms, you will design a system that analyzes facial and bone structure features and emotions, and compares them with a database of fashion models to generate advice."

[0286] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0287] Step 1:

[0288] Users take or select a photo of their face and bone structure and upload it to a server through the app.

[0289] Specifically, the user launches the app and taps the "Take a Photo" button. The user then uses the camera to take a photo of their face and whole body, and taps the "Upload" button afterward. The input is the user's photo data, and the output is the photo data to be sent to the server.

[0290] Step 2:

[0291] The server receives the photo data uploaded by the user and temporarily stores it.

[0292] Specifically, the server receives photo data via an HTTP request and temporarily stores it in a database. The input is photo data from the user's device, and the output is the image data stored on the server.

[0293] Step 3:

[0294] The server analyzes the stored photographic data and extracts facial and skeletal features.

[0295] Specifically, the server uses OpenCV to perform facial recognition and extract skeletal features. It applies image processing algorithms to detect facial contours and skeletal points. The input is temporarily saved image data, and the output is facial and skeletal feature data.

[0296] Step 4:

[0297] The emotion engine analyzes facial expression data and recognizes specific emotions.

[0298] Specifically, facial expression data is fed into a machine learning algorithm such as "DeepFace" to recognize emotions (e.g., "happiness" or "sadness"). The input is facial expression data, and the output is emotional data.

[0299] Step 5:

[0300] The server compares the extracted facial and skeletal feature data and emotion data with a database of fashion models.

[0301] Specifically, the server issues a query to the fashion model database to retrieve data on models with similar features and emotions. The input is feature data and emotion data, and the output is the optimal model data.

[0302] Step 6:

[0303] The server generates specific fashion advice based on the acquired model data and the user's emotional data.

[0304] Specifically, the server analyzes the recommended fashion items and styling details based on the acquired model data, and generates additional suggestions based on the user's emotions. The inputs are model data and emotion data, and the output is generated advice data.

[0305] Step 7:

[0306] The generated advice data is sent to the user terminal, and the user confirms the advice through the app.

[0307] Specifically, the server sends the generated advice data to the user device via the endpoint. The user device receives the advice data from the endpoint and displays it on the app interface. The input is the generated advice data, and the output is the advice displayed on the user device.

[0308] (Application example 2)

[0309] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0310] Conventional fashion advice systems suggest styles based on the user's facial and skeletal features, but are unable to consider the user's emotional state. This makes it difficult to provide more personalized advice tailored to the user's mood. It is also difficult to provide real-time fashion advice in physical stores. Therefore, there is a need for the development of a system that can consider the user's emotions and provide optimal fashion advice in real time.

[0311] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0312] In this invention, the server includes means for allowing a user to upload a photograph of their face and skeletal structure, means for receiving the uploaded photograph and extracting facial and skeletal features and emotions, means for comparing the extracted features and emotions with a fashion model database to calculate the most suitable fashion for the user, means for generating advice on the most suitable fashion items and style and sending it to the user terminal together with additional suggestions based on the emotions, and means for displaying the generated advice on a user interface and a display device, thereby enabling the provision of personalized fashion advice that takes the user's emotions into consideration.

[0313] "Means for enabling users to upload photographs of their own face and bones" refers to a function for taking or selecting photographs of the face and bones using the user terminal and sending them to the server.

[0314] "Means for the server to receive uploaded photos and extract facial and skeletal features and emotions" refers to technology in which the server analyzes image data sent from the user terminal and identifies the facial shape, skeletal features, and current emotional state.

[0315] "Means for comparing the extracted features and emotions with a database of fashion models and calculating the most suitable fashion for the user" refers to a function that recommends the most suitable fashion style and items from a database of fashion models based on facial and skeletal features and emotions.

[0316] "Means for generating advice regarding optimal fashion items and styles and transmitting the advice to a user terminal together with additional suggestions based on emotions" refers to a technology for generating additional suggestions tailored to the user's emotional state along with fashion advice and transmitting the same from a server to a user terminal.

[0317] "Means for displaying generated advice on a user interface and display device" refers to functionality for visually presenting generated fashion advice and emotion-based suggestions on a user terminal and other display devices.

[0318] "Means using image processing technology and machine learning algorithms" refers to technology that uses image processing software and machine learning models to analyze image data and perform accurate feature extraction and emotion recognition.

[0319] A "fashion model database" refers to a data storage that accumulates information on various face shapes, body types, fashion items, and styles.

[0320] This system allows users to upload photos of their face and bone structure in a physical store, and provides fashion advice based on the user's emotions. The system consists of a user terminal, a server, an emotion engine, and a fashion model database.

[0321] Overall system configuration

[0322] User device functions

[0323] The user device is equipped with an application that allows users to take and select photos and upload them to a server. It also has the function of displaying fashion advice sent from the server and personalized recommendations based on emotions. Specifically, this corresponds to smart glasses or head-mounted displays installed in fitting rooms, etc.

[0324] Server Features

[0325] The server receives photos and facial expression data uploaded from the user's device and analyzes facial and skeletal features and emotions. The analysis uses image processing techniques (e.g., OpenCV), machine learning algorithms (e.g., TensorFlow / Keras), and an emotion engine (EmotionEngine). After analysis, the server compares the results with a database of fashion models and calculates advice based on the user's optimal fashion items, style, and emotions. The final advice is sent to the user's device.

[0326] Emotion Engine Functions

[0327] The Emotion Engine is a module that recognizes emotions from facial expression data uploaded by users. This engine uses machine learning algorithms to analyze the user's facial expressions and identify the emotion at that time. For example, if the Emotion Engine recognizes the emotion "joy" from the user's facial expression, it will recommend fashion items that match that emotion, including a relaxed style and cheerful color palette.

[0328] Fashion Model Database Features

[0329] This database stores information on fashion models with various facial shapes and skeletal features. The database includes information on each model's facial shape, body type, recommended fashion style, and item list. Specifically, if a model with a "round face and plump body" is registered, the style and items recommended by that model (e.g., A-line dresses, flared pants, etc.) will be recorded.

[0330] Fashion advice generation and transmission to user devices

[0331] After the server selects the optimal model, it generates specific fashion advice based on the model's data and the user's emotions. The advice includes recommended fashion items, styling details, and emotional suggestions. The generated advice is sent to the user's device and displayed on a user interface.

[0332] Explanation of program processing

[0333] 1. Hardware and Software Use

[0334] Camera (e.g. Logitech C920): Takes a photo of the user.

[0335] Emotion Engine: Analyze emotions using machine learning (e.g. TensorFlow / Keras).

[0336] Fashion Advice Server: A server that analyzes facial features and bone structure (e.g., built with Flask).

[0337] Fashion Model Database: Provides fashion model information based on facial and skeletal features.

[0338] Display device (e.g., smart glasses, HMD): displays advice to the user (e.g., Google Glass, HoloLens).

[0339] 2. Data processing or data calculation

[0340] Image Capture: Take a picture of the user with the camera and store it locally.

[0341] Image analysis: Sends the image to an analysis server to analyze the facial shape, bone structure, and emotion.

[0342] Database matching: The analysis results are compared with a fashion model database to select the most suitable model and style.

[0343] Advice generation: Generate personalized fashion advice based on the analysis results and information obtained from the model.

[0344] Adding specific examples

[0345] For example, if a user's facial expression captured by a "smart mirror" is "surprised," their face shape is "round," and their body type is "slim," the device will display the following message in real time:

[0346] "If you have a round face, a V-neck top is a great choice. A surprised look is great, so add some lighter colors to make it stand out even more."

[0347] Example prompts to input to the generative AI model

[0348] Generate optimal fashion advice for a user with a round face, slim figure, and surprised expression.

[0349] Create a system that provides real-time fashion advice based on facial shape and emotion.

[0350] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0351] Step 1:

[0352] The user takes a photo.

[0353] Input: The user takes a photo of their face and skeletal structure using smart glasses or a head-mounted display.

[0354] Data processing: The camera captures a photo of the user's face and overall appearance, and stores it locally as an image file.

[0355] Output: The saved image file (e.g. user_image.jpg).

[0356] Step 2:

[0357] The user terminal uploads the photo to the server.

[0358] Input: A saved image file.

[0359] Data processing: The user device sends the image file to the server.

[0360] Output: Image data sent to the server.

[0361] Step 3:

[0362] The server receives the image data and extracts facial and skeletal features and emotions.

[0363] Input: Image data sent to the server.

[0364] Data processing:

[0365] Step 1: The server uses image processing technology (e.g., OpenCV) to analyze the facial shape and skeletal features.

[0366] Step 2: The server uses a machine learning algorithm (e.g., TensorFlow / Keras) to identify the emotional state (e.g., "surprise," "joy," etc.) from the image.

[0367] Output: Extracted face shape, skeletal features, and emotion data.

[0368] Step 4:

[0369] The server compares the extracted features and emotions with a fashion model database and calculates the most suitable fashion.

[0370] Input: Extracted face shape, skeletal features, and emotion data.

[0371] Data processing:

[0372] Operation 1: The server accesses a fashion model database (e.g., FashionModelDatabase) and searches for the corresponding model data.

[0373] Action 2: Based on the collated data, the best fashion items and styles are selected for the user.

[0374] Output: Data about the best fashion items and styles.

[0375] Step 5:

[0376] The server generates advice and sends it to the user terminal.

[0377] Input: Data about optimal fashion items and styles, and sentiment data.

[0378] Data processing:

[0379] Action 1: The server generates fashion advice, which includes selected fashion items and styles, as well as additional suggestions based on the user's emotions.

[0380] Operation 2: The generated advice is sent to the user terminal.

[0381] Output: Fashion advice sent to the user's device.

[0382] Step 6:

[0383] The user terminal displays the generated advice.

[0384] Input: Fashion advice sent to user device.

[0385] Data processing: Displaying the generated advice on a user interface and display device (e.g., smart glasses, HMD).

[0386] Output: Fashion advice that the user can visually see.

[0387] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0388] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0389] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0390] [Second embodiment]

[0391] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0392] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0393] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0394] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0395] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0396] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0397] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0398] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0399] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0400] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0401] In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0402] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0403] Overall system configuration

[0404] This invention is a system that allows users to upload photos of their face and bone structure and provides fashion advice based on those photos. The system is composed of a user terminal, a server, and a database of fashion models.

[0405] User device functions

[0406] The user device has an application that allows users to take and select photos and upload them to the server, and also has the function of displaying fashion advice sent from the server.

[0407] Example: A user launches an app, uses the camera to take a photo of their face and body, and then uploads the photo to a server through the app's interface.

[0408] Server Features

[0409] The server receives photos uploaded from the user's device and analyzes facial and skeletal features using image processing technology and machine learning algorithms. After analysis, the photos are compared with a database of fashion models to calculate the most suitable fashion items and style for the user. Final advice is then sent to the user's device.

[0410] Example: The server receives the uploaded photo and applies a facial recognition algorithm to extract facial shape and skeletal features. The extracted data is compared with a database to select the most suitable fashion model. For example, if the user's characteristics are "long face, slim figure," the server will search the database for a model with the closest characteristics.

[0411] Fashion Model Database Features

[0412] This database stores information on fashion models with various facial shapes and bone structures. The database includes each model's facial shape, body shape, recommended fashion style, and item list.

[0413] Example: If a model with a "round face and plump figure" is registered in the database, the style and items recommended by that model (e.g., A-line dresses, flared pants, etc.) will be recorded.

[0414] Fashion advice generation and transmission to user devices

[0415] After the server selects the optimal model, it generates specific fashion advice based on the model's data. The advice includes recommended fashion items and styling details. The advice is then sent to the user's device and displayed on a user interface.

[0416] Example: Based on the user's characteristics, specific styling advice is generated, such as "V-neck tops and high-waisted pants look good on you." This advice is sent to the user's device and displayed within the app.

[0417] Use Case Details

[0418] For example, if a user has a "long face and slim figure," the server extracts the corresponding features from the database and determines that "Model A" is the best fit. Model A recommends that "V-neck tops and high-waisted pants look good on you," and this styling advice is provided to the user. The user can check this advice within the app and use it as a reference for shopping and coordinating outfits.

[0419] The system allows users to receive accurate fashion advice based on their facial and bone structure, making it easy to find the style that best suits them.

[0420] The processing flow will be explained below.

[0421] Step 1:

[0422] The user launches the smartphone app and takes a photo of their face and whole body, or selects an existing photo.

[0423] Step 2:

[0424] Users upload photos to the server through the app's interface, and the photo data is transmitted over secure communication.

[0425] Step 3:

[0426] The server receives photo data uploaded by the user.

[0427] Step 4:

[0428] The server passes the received photo data to the image processing module for analysis, where the face detection and feature extraction process begins.

[0429] Step 5:

[0430] The server's image processing module extracts facial shape and skeletal features, such as facial contours, eye, nose, and mouth positions, and skeletal proportions.

[0431] Step 6:

[0432] The server then compares the extracted feature data with a database of fashion models, which contains data on models with various facial and body shapes.

[0433] Step 7:

[0434] The server searches the database for a model that best matches the user's characteristics and calculates the optimal fashion style based on that model.

[0435] Step 8:

[0436] The server generates advice on optimal fashion items and styles, including specific clothing types and styling suggestions.

[0437] Step 9:

[0438] The server transmits the generated fashion advice to the user's terminal.

[0439] Step 10:

[0440] The user's device receives the advice data sent from the server and displays it within the application, allowing the user to check it and use it to help them choose their own fashion.

[0441] Example 1

[0442] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0443] Conventional fashion advice systems often have difficulty providing personalized advice that takes into account the user's facial and body shapes. Furthermore, they have faced the problem of requiring time and effort for users to find the fashion items and styling that best suit them. This makes it difficult for users to easily find the style that best suits them.

[0444] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0445] In this invention, the server includes means for allowing users to upload photos of their face and skeletal structure, means for receiving the uploaded photos and extracting facial and skeletal features, means for comparing the extracted features with a fashion model database to calculate the most suitable fashion for the user, means for generating advice on the most suitable fashion items and style and sending it to the user terminal, means for displaying the generated advice on a user interface, and means for performing a facial recognition algorithm, a machine learning algorithm, and database comparison. This allows users to receive accurate fashion advice based on their facial and skeletal features, making it easy to find the style that best suits them.

[0446] "User Device" means a device equipped with an application that allows a user to take and select photos and upload them to the server, and that has the functionality to display fashion advice sent from the server.

[0447] The "server" is a central processing unit that receives photos uploaded from the user's terminal, analyzes facial and skeletal features, compares the analysis results with a database of fashion models to calculate the most suitable fashion items and style for the user, and sends the generated advice to the user's terminal.

[0448] A "fashion model database" is a database that stores information on fashion models with various facial shapes and skeletal features, and is a data storage that includes each model's facial shape, body shape, recommended fashion style, item list, etc.

[0449] A "facial recognition algorithm" is an algorithm used to analyze the shape and features of a face from an uploaded photo, and is a technology that uses image processing libraries such as OpenCV and Dlib to detect facial contours and feature points.

[0450] A "machine learning algorithm" is an algorithm that performs classification and prediction based on extracted facial and skeletal feature data, and is a technology that analyzes feature data using methods such as Random Forest and SVM (Support Vector Machine).

[0451] "Fashion items" are specific clothing, accessories, and other fashion products that are suggested to a user and are recommended based on the user's facial and skeletal features.

[0452] A "prompt" is an instruction entered into a generative AI model to output appropriate fashion advice, and is text input to obtain optimal output based on the user's characteristics and requests.

[0453] MODE FOR CARRYING OUT THE INVENTION

[0454] Overall system configuration

[0455] This invention is a system that allows users to upload photos of their face and bone structure and provides fashion advice based on those photos. The system is composed of a user terminal, a server, and a database of fashion models.

[0456] User device functions

[0457] The user device has an application that allows users to take and select photos and upload them to the server. It also has a function to display fashion advice sent from the server. Users launch the app, take photos of their face and body using the camera, and then upload the photos to the server through the app interface.

[0458] Examples:

[0459] The user launches the app on their smartphone, takes a photo of their face and whole body, and then clicks the app's upload button to send the photo to the server.

[0460] Server Features

[0461] The server receives photos uploaded from the user's device and analyzes facial and skeletal features. This analysis uses image processing technology and machine learning algorithms. Specifically, OpenCV and Dlib are used to detect facial shape and feature points, and the data is analyzed using machine learning algorithms such as Random Forest and SVM. After analysis, the results are compared with a fashion model database to calculate the optimal fashion items and style for the user. Final advice is then sent to the user's device.

[0462] Examples:

[0463] The server receives the photo sent by the user and performs facial recognition using OpenCV. It extracts facial feature points using Dlib and then applies the Random Forest algorithm to identify the user's facial and skeletal characteristics. It then compares the results with a fashion model database to select a model that most closely matches the "long face, slim body shape" description and calculates the model's recommended fashion style.

[0464] Fashion Model Database Features

[0465] This database stores information on fashion models with various facial shapes and bone structures. The database includes each model's facial shape, body shape, recommended fashion style, and item list.

[0466] Examples:

[0467] If a model with a "round face and plump figure" is registered in the database, the style and items that model recommends (e.g., A-line dresses, flared pants, etc.) are recorded.

[0468] Fashion advice generation and transmission to user devices

[0469] After the server selects the optimal model, it generates specific fashion advice based on the model's data. The advice includes recommended fashion items and styling details. The advice is sent to the user's device and displayed on a user interface.

[0470] Examples:

[0471] Based on the user's characteristics, specific styling advice is generated, such as "V-neck tops and high-waisted pants look good on you." This advice is sent to the user's device and displayed within the app.

[0472] Prompt Sentence Examples

[0473] A typical example of a prompt to be input to a generative AI model might be:

[0474] "Based on photos of the user's face and body, please suggest the most suitable fashion items and styles. The user's characteristics are as follows: 'Long face, slim body'. Please output the most suitable fashion advice for this."

[0475] Using this prompt, the generative AI model outputs content that provides appropriate fashion advice.

[0476] The system allows users to receive accurate fashion advice based on their facial and bone structure, making it easy to find the style that best suits them.

[0477] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0478] Step 1:

[0479] Receiving and storing photos on the server

[0480] The server receives photos uploaded from the user's device. This input is a face and full-body photo sent by the user through the app. The server saves the received photos in temporary storage. As a concrete example, the server receives image data through an HTTP request and saves the data in local storage.

[0481] Input: Face and full-body photo data sent from the user device.

[0482] Data processing: Analyzing HTTP requests and saving image data.

[0483] Output: Saved image file.

[0484] Step 2:

[0485] Server-based face recognition and skeletal feature extraction

[0486] The server analyzes the stored photos to extract facial shape and skeletal features. This step uses image processing technologies such as OpenCV and Dlib. First, a facial recognition algorithm is applied, and then Dlib is used to extract the coordinates of facial feature points (eyes, nose, mouth, etc.).

[0487] Input: Saved face and full-body image files.

[0488] Data processing: Face recognition using OpenCV and feature point extraction using Dlib.

[0489] Output: Facial shape and skeletal feature coordinate data.

[0490] Step 3:

[0491] Application of machine learning algorithms by the server

[0492] The server inputs the extracted facial and skeletal feature data into a machine learning algorithm to classify the user's face shape and body shape. The algorithms used include Random Forest and SVM. In this step, the feature data is input into the machine learning model as a numerical vector to obtain the classification results.

[0493] Input: Coordinate data of facial shape and skeletal features.

[0494] Data processing: Vectorizing feature data and inputting it into a machine learning model.

[0495] Output: Classified face shape and body shape data.

[0496] Step 4:

[0497] Database verification by the server

[0498] The server compares the classified face shape and body shape data with a fashion model database, which contains each model's face shape, body shape, recommended fashion style, and items. In this step, the server searches for the most suitable model and obtains the model's data.

[0499] Input: Classified face shape and body shape data.

[0500] Data processing: Search the fashion model database and select the most suitable model.

[0501] Output: Data of the best fashion model.

[0502] Step 5:

[0503] Server-based fashion advice generation

[0504] The server generates specific fashion advice based on the data of the selected fashion model. Using the generative AI model, it generates text that suggests the most suitable fashion items and styles for the user. For example, it might generate advice such as "V-neck tops and high-waisted pants look good on you."

[0505] Input: Data of the optimal fashion model.

[0506] Data processing: Generate text advice using a generative AI model.

[0507] Output: Text data of fashion advice.

[0508] Step 6:

[0509] Server sends advice and displays it on the user's device

[0510] The server sends the generated fashion advice to the user's device, which then displays the received advice within the application. Specifically, the server sends the advice as an HTTP response, and the user's device analyzes it and displays it on the interface.

[0511] Input: Text data of fashion advice.

[0512] Data processing: Sending advice and analyzing received data.

[0513] Output: User interface with advice displayed.

[0514] (Application example 1)

[0515] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0516] Fashion-conscious users face challenges in easily finding the styles and items that best suit them. In particular, there are few methods for quickly providing personalized fashion advice based on a user's face shape and bone structure, making it time-consuming and labor-intensive to select a style that suits them. Furthermore, there is no way to immediately purchase products based on the advice provided, which can disrupt the user's shopping experience. To address these issues, a system is needed that can quickly and accurately analyze a user's characteristics, provide optimal fashion advice, and enable the user to purchase products based on the advice.

[0517] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0518] In this invention, the server includes means for allowing a user to upload images of their face and bone structure, means for receiving the uploaded images and extracting facial and bone structure features, means for comparing the extracted features with a fashion model database to calculate the most suitable fashion for the user, means for generating advice on the most suitable fashion items and style and transmitting it to the user terminal, means for displaying the generated advice on a user interface, and means for allowing the user to purchase products based on the advice through an application. This allows the user to receive the most suitable fashion advice based on their own features and instantly purchase products based on the advice.

[0519] A "user terminal" is a device that allows a user to take photos, upload images to a server through an application, and receive fashion advice.

[0520] The "server" is a computer system that receives images sent from a user terminal, analyzes them, generates fashion advice, and sends it to the user terminal.

[0521] "Image processing technology" is a technology for analyzing images taken by a user and extracting facial and skeletal features.

[0522] "Machine learning algorithms" are artificial intelligence technologies that compare facial and skeletal features with a database to generate optimal fashion advice for users.

[0523] A "fashion model database" is a collection of information that stores data on models based on various face shapes and body shapes, and is used to provide fashion advice based on that data.

[0524] The "user interface" refers to a screen or operating means for displaying the generated fashion advice to the user.

[0525] A "prompt sentence" is text data that uses a generative AI model to interactively analyze the user's image and features.

[0526] The "shopping function" is a function that allows a user to directly purchase items shown in the advice based on the generated fashion advice.

[0527] This invention is a system that allows users to upload images of their face and bone structure and provides fashion advice based on those images. The system consists of a user terminal, a server, and a database of fashion models.

[0528] User device functions

[0529] The user terminal has a function that allows the user to take pictures and upload them to the server through the application, and also provides an interface that displays fashion advice sent from the server and allows the user to purchase products based on that advice.

[0530] For example, a user launches the app and takes a picture of their face and whole body using their camera. They then upload the image to a server through the app's interface. After uploading, the user waits for a response from the server and can view the resulting advice on the screen. Based on this advice, a link or button to purchase the product is provided, allowing the user to continue shopping.

[0531] Server Features

[0532] The server receives the images sent from the user terminal and extracts facial and skeletal features using image processing techniques and machine learning algorithms.

[0533] Specifically, the server does the following:

[0534] 1. Image reception: Has an API endpoint for receiving image files sent from the user device.

[0535] 2. Feature extraction: The received image is analyzed and facial and skeletal features are extracted using a facial recognition algorithm, such as the Python face_recognition library. Numerical data such as facial shape and skeletal proportions are obtained.

[0536] 3. Database Matching: The extracted features are matched against a database of fashion models, which contains feature data for each model and fashion styles based on those features.

[0537] 4. Advice Generation: Select the best model and generate styling advice based on that model. This advice is crafted in detail using the generative AI model and prompt text.

[0538] 5. Sending advice: Install an API to send the generated advice to the user's device.

[0539] Fashion Model Database Features

[0540] The fashion model database stores information on models with various facial shapes and bone structures. The database includes each model's facial shape, body shape, recommended fashion style, and item list.

[0541] For example, if a model with a "round face and plump figure" is registered in the database, the style and clothing that model recommends (e.g., A-line dresses, flared pants, etc.) are recorded. This allows the server to provide advice that best suits the user's characteristics.

[0542] Fashion advice generation and transmission to user devices

[0543] After the server selects the optimal model, it generates specific fashion advice based on the model's data. The advice includes recommended fashion items and styling details. The advice is sent to the user's device and displayed on a user interface.

[0544] For example, styling advice such as "V-neck tops and high-waisted pants look good on you" is generated based on the user's characteristics. This advice is sent to the user's device and displayed within the app. The user can also purchase items directly within the app.

[0545] Prompt Sentence Examples

[0546] "Analyze the user's photo and extract facial shape and bone structure features. Based on those features, provide the most suitable fashion items and styling advice."

[0547] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0548] Step 1:

[0549] The user launches the smartphone app and takes a photo of their face and whole body.

[0550] In this step, the user takes a photo using the app's camera function and the photo is saved in the smartphone. The input is the original photo taken by the user, and the output is the photo data saved in the smartphone.

[0551] Step 2:

[0552] The user terminal uploads the photograph to the server.

[0553] In this step, when a user clicks the upload button in the app, the photo file is sent to the server through the HTTP request process within the app. The input is the photo data stored on the smartphone, and the output is the data sent to the server.

[0554] Step 3:

[0555] The server receives the photo data and uses image processing techniques and machine learning algorithms to extract facial and skeletal features.

[0556] Specifically, the server analyzes the photo using Python's face_recognition library and extracts the facial shape and skeletal features as numerical data. The input is the photo data sent to the server, and the output is the extracted facial and skeletal feature data.

[0557] Step 4:

[0558] Based on the extracted feature data, the server compares it with a database of fashion models and calculates the most suitable fashion for the user.

[0559] The server uses the extracted feature data to compare it with the features of each model stored in the database and selects the optimal model. The input is the extracted feature data and a database of fashion models, and the output is the optimal model data and styling advice.

[0560] Step 5:

[0561] The server uses the generated AI model and prompts to generate specific fashion advice.

[0562] In this step, the server inputs prompts into the generative AI model to generate specific fashion item and style advice based on the user's characteristics. The input is the optimal model's data and prompts, and the output is specific fashion advice.

[0563] Step 6:

[0564] The server transmits the generated advice to the user terminal.

[0565] The server sends the generated advice to the user terminal as an HTTP response. The input is the specific fashion advice, and the output is the advice data sent to the user terminal.

[0566] Step 7:

[0567] The user terminal displays the received advice on a user interface, and the user purchases the product based on the advice.

[0568] In this step, the user can review the advice displayed within the app and purchase the product by clicking a purchase link or button. The input is the advice data received from the server, and the output is the advice and purchase procedure displayed on the user's display screen.

[0569] Prompt Sentence Examples

[0570] "Analyze the user's photo and extract facial shape and bone structure features. Based on those features, provide the most suitable fashion items and styling advice."

[0571] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0572] Overall system configuration

[0573] This invention is a system that allows users to upload photos of their face and bone structure, and provides fashion advice based on the user's emotions. The system consists of a user terminal, a server, an emotion engine, and a database of fashion models.

[0574] User device functions

[0575] The user device is equipped with an application that allows users to take and select photos and upload them to the server, and also has the function of displaying fashion advice and personalized recommendation advice based on emotions sent from the server.

[0576] Example: A user launches an app, uses the camera to take a photo of their face and body, captures facial expressions to recognize their current emotion, and then uploads the photo to a server through the app interface.

[0577] Server Features

[0578] The server receives photos and facial expression data uploaded from the user's device and analyzes facial and skeletal features and emotions. The analysis uses image processing technology, machine learning algorithms, and an emotion engine. After analysis, the server compares the results with a database of fashion models and calculates advice based on the user's optimal fashion items, style, and emotions. The final advice is then sent to the user's device.

[0579] Example: The server receives the uploaded photo and facial expression data, and applies facial recognition and emotion recognition algorithms to extract facial shape, skeletal features, and the current emotional state (e.g., "joy" or "sadness"). The extracted data is compared with a database to select the most suitable fashion model.

[0580] Emotion Engine Functions

[0581] The emotion engine is a module for recognizing emotions from facial expression data uploaded by users. This engine uses machine learning algorithms to analyze the user's facial expressions and identify the emotion at that time.

[0582] Example: If the emotion engine recognizes the emotion "joy" from a user's facial expression, it will recommend fashion items that match that emotion, including relaxed styles and cheerful color palettes.

[0583] Fashion Model Database Features

[0584] This database stores information on fashion models with various facial shapes and bone structures. The database includes each model's facial shape, body shape, recommended fashion style, and item list.

[0585] Example: If a model with a "round face and plump figure" is registered in the database, the style and items recommended by that model (e.g., A-line dresses, flared pants, etc.) will be recorded.

[0586] Fashion advice generation and transmission to user devices

[0587] After the server selects the optimal model, it generates specific fashion advice based on the model's data and the user's emotions. The advice includes recommended fashion items, styling details, and emotional suggestions. The generated advice is sent to the user's device and displayed on a user interface.

[0588] Example: Based on the user's characteristics and emotions, specific styling advice such as "V-neck tops and high-waisted pants look good on you" is provided, along with additional emotional advice such as "Bright colors are recommended for today's mood." This advice is sent to the user's device and displayed within the app.

[0589] Use Case Details

[0590] For example, if a user has a "long face, slim figure," and is currently feeling "happy," the server will extract the corresponding features and emotions from the database and determine that "Model A" is the best fit. Model A recommends that "V-neck tops and high-waisted pants look good on you," so in addition to this styling advice, it will provide the user with emotion-based advice such as "bright-colored items are perfect for your mood today." Users can check these advice within the app and use them as reference for shopping and outfits.

[0591] The system allows users to receive accurate fashion advice based on their facial and skeletal features and emotions, making it easy to find the style that best suits them.

[0592] The processing flow will be explained below.

[0593] Step 1:

[0594] The user launches the smartphone app and takes a photo of their face, their whole body, and their facial expression, or selects an existing photo.

[0595] Step 2:

[0596] Users upload photo data and facial expression data to the server through the app interface, and the data is transmitted over secure communication.

[0597] Step 3:

[0598] The server receives the photo data and facial expression data uploaded by the user.

[0599] Step 4:

[0600] The server then passes the received photo data to the image processing module for analysis, where the process of recognizing facial and skeletal features and facial expressions begins.

[0601] Step 5:

[0602] The server's image processing module extracts facial shape and skeletal features, such as facial contours, eye, nose, and mouth positions, and skeletal proportions.

[0603] Step 6:

[0604] Furthermore, an emotion engine analyzes the facial expression data to identify the user's current emotion, such as "happiness," "sadness," or "surprise."

[0605] Step 7:

[0606] The server compares the extracted feature data and emotion data with a database of fashion models, which contains data on models with various facial and body shapes.

[0607] Step 8:

[0608] The server searches the database for a model that best matches the user's facial and skeletal features and emotional data, and calculates the optimal fashion style based on that model.

[0609] Step 9:

[0610] The server generates advice about the calculated fashion items and styles, including recommended clothing types, styling details, and additional suggestions depending on the emotion.

[0611] Step 10:

[0612] The server sends the generated fashion advice to the user's device, including colors and styling that match the user's emotions.

[0613] Step 11:

[0614] The user's device receives the advice data sent from the server and displays it within the application, allowing the user to check it and use it to help them choose their own fashion.

[0615] Step 12:

[0616] Based on the advice provided, the user purchases the recommended fashion items or puts together an outfit.

[0617] Example 2

[0618] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0619] Conventional fashion advice systems only consider the user's facial and skeletal features, making it difficult to provide advice that reflects the user's current emotions. Furthermore, advice that does not consider emotions makes it difficult to select the optimal fashion that matches the user's feelings. Therefore, there is a need for a method to support users in selecting fashion that suits their daily mood and the occasion.

[0620] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0621] In this invention, the server includes a means for allowing a user to upload a photograph of their face and skeletal structure, a means for receiving the uploaded photograph and extracting facial and skeletal features, and a means for analyzing the user's emotions using an emotion engine, thereby enabling optimal fashion advice to be provided to the user based on the extracted features and analyzed emotions.

[0622] "User" refers to any person who uses the system to upload their own photos and receive fashion advice.

[0623] "Server" refers to a computer system that receives data uploaded by users and performs analysis and advice generation.

[0624] "Photo" refers to image data including a user's face and bone structure that is uploaded by the user.

[0625] "Facial and skeletal features" refers to data being analyzed that indicates the user's physical characteristics, such as facial shape and skeletal structure.

[0626] "Emotion engine" refers to a module that analyzes and identifies emotions from a user's facial expression data.

[0627] A "fashion model database" refers to an information repository that stores data on each model based on their face shape and body type.

[0628] "Means for generating advice" refers to the process of creating recommendations for the most suitable fashion items and styles for the user based on the analyzed features and emotional data.

[0629] "User Terminal" refers to the device used by a User to upload photos and receive and view advice.

[0630] "User interface" refers to the screen and interaction mechanism that displays advice generated on the user's terminal and allows the user to operate it.

[0631] "Image processing techniques" refers to techniques used to extract facial and skeletal features from photographs.

[0632] "Machine learning algorithms" refer to computational techniques used for data analysis and emotion recognition.

[0633] The system of this invention allows users to upload photos of their face and bone structure, and provides appropriate fashion advice based on the user's emotions. The system is composed of a user terminal, a server, an emotion engine, and a database of fashion models.

[0634] User device functions

[0635] The user device is equipped with an application that allows users to take and select photos and upload them to the server, and also has the function of displaying fashion advice and personalized recommendation advice based on emotions sent from the server.

[0636] Specifically, the user launches the app and takes a photo of their face and whole body using the camera. Furthermore, facial expressions are captured to recognize the current emotion, and the captured images are uploaded to a server through the app's interface. The hardware used is a mobile device such as a smartphone or tablet, and the app is developed based on a general-purpose application such as a "movie viewer for mobile devices."

[0637] Server Features

[0638] The server receives photos and facial expression data uploaded from the user's device and analyzes facial and skeletal features and emotions. Image processing technology, image processing libraries such as "OpenCV," and machine learning algorithms such as "DeepFace" are used for the analysis. The server then compares the analysis results with a database of fashion models to calculate advice based on the user's optimal fashion items, style, and emotions. The final advice is then sent to the user's device.

[0639] Specifically, the server receives a photo and facial expression data from the user, extracts facial shape and skeletal features, and simultaneously recognizes the user's emotional state (e.g., "joy" or "sadness"). The extracted data is compared with a database of fashion models, and suggestions for optimal fashion items, styling details, and emotions are generated. This data is then sent to the user's device, where the user can view it within the app.

[0640] Emotion Engine Functions

[0641] The emotion engine is a module for recognizing emotions from facial expression data uploaded by users. This engine uses machine learning algorithms to analyze the user's facial expressions and recognize specific emotions.

[0642] For example, if the emotion engine recognizes the emotion of "joy" from the user's facial expression, it will recommend fashion items that match that emotion, including a relaxed style and cheerful color palette. The machine learning algorithms used can be "TensorFlow" or "Keras."

[0643] Fashion Model Database Features

[0644] The fashion model database stores information on fashion models with various facial shapes and skeletal features. The database includes each model's facial shape, body shape, recommended fashion style, and item list.

[0645] Specifically, if a model with a "round face and plump figure" is registered in the database, the style and items recommended by that model (e.g., A-line dress, flared pants, etc.) are recorded.

[0646] Fashion advice generation and transmission to user devices

[0647] After the server selects the optimal model, it generates specific fashion advice based on the model's data and the user's emotions. The advice includes recommended fashion items, styling details, and emotional suggestions. The generated advice is sent to the user's device and displayed on a user interface.

[0648] As a specific example, based on the user's characteristics and emotions, in addition to specific styling advice such as "V-neck tops and high-waisted pants look good on you," additional emotional advice such as "bright colors are recommended for today's mood" is provided.

[0649] Prompt Sentence Examples

[0650] "You will create a program that allows users to upload photos of their face and bone structure, recognizes emotions, and provides optimal fashion advice. Using image processing technology and machine learning algorithms, you will design a system that analyzes facial and bone structure features and emotions, and compares them with a database of fashion models to generate advice."

[0651] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0652] Step 1:

[0653] Users take or select a photo of their face and bone structure and upload it to a server through the app.

[0654] Specifically, the user launches the app and taps the "Take a Photo" button. The user then uses the camera to take a photo of their face and whole body, and taps the "Upload" button afterward. The input is the user's photo data, and the output is the photo data to be sent to the server.

[0655] Step 2:

[0656] The server receives the photo data uploaded by the user and temporarily stores it.

[0657] Specifically, the server receives photo data via an HTTP request and temporarily stores it in a database. The input is photo data from the user's device, and the output is the image data stored on the server.

[0658] Step 3:

[0659] The server analyzes the stored photographic data and extracts facial and skeletal features.

[0660] Specifically, the server uses OpenCV to perform facial recognition and extract skeletal features. It applies image processing algorithms to detect facial contours and skeletal points. The input is temporarily saved image data, and the output is facial and skeletal feature data.

[0661] Step 4:

[0662] The emotion engine analyzes facial expression data and recognizes specific emotions.

[0663] Specifically, facial expression data is fed into a machine learning algorithm such as "DeepFace" to recognize emotions (e.g., "happiness" or "sadness"). The input is facial expression data, and the output is emotional data.

[0664] Step 5:

[0665] The server compares the extracted facial and skeletal feature data and emotion data with a database of fashion models.

[0666] Specifically, the server issues a query to the fashion model database to retrieve data on models with similar features and emotions. The input is feature data and emotion data, and the output is the optimal model data.

[0667] Step 6:

[0668] The server generates specific fashion advice based on the acquired model data and the user's emotional data.

[0669] Specifically, the server analyzes the recommended fashion items and styling details based on the acquired model data, and generates additional suggestions based on the user's emotions. The inputs are model data and emotion data, and the output is generated advice data.

[0670] Step 7:

[0671] The generated advice data is sent to the user terminal, and the user confirms the advice through the app.

[0672] Specifically, the server sends the generated advice data to the user device via the endpoint. The user device receives the advice data from the endpoint and displays it on the app interface. The input is the generated advice data, and the output is the advice displayed on the user device.

[0673] (Application example 2)

[0674] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0675] Conventional fashion advice systems suggest styles based on the user's facial and skeletal features, but are unable to consider the user's emotional state. This makes it difficult to provide more personalized advice tailored to the user's mood. It is also difficult to provide real-time fashion advice in physical stores. Therefore, there is a need for the development of a system that can consider the user's emotions and provide optimal fashion advice in real time.

[0676] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0677] In this invention, the server includes means for allowing a user to upload a photograph of their face and skeletal structure, means for receiving the uploaded photograph and extracting facial and skeletal features and emotions, means for comparing the extracted features and emotions with a fashion model database to calculate the most suitable fashion for the user, means for generating advice on the most suitable fashion items and style and sending it to the user terminal together with additional suggestions based on the emotions, and means for displaying the generated advice on a user interface and a display device, thereby enabling the provision of personalized fashion advice that takes the user's emotions into consideration.

[0678] "Means for enabling users to upload photographs of their own face and bones" refers to a function for taking or selecting photographs of the face and bones using the user terminal and sending them to the server.

[0679] "Means for the server to receive uploaded photos and extract facial and skeletal features and emotions" refers to technology in which the server analyzes image data sent from the user terminal and identifies the facial shape, skeletal features, and current emotional state.

[0680] "Means for comparing the extracted features and emotions with a database of fashion models and calculating the most suitable fashion for the user" refers to a function that recommends the most suitable fashion style and items from a database of fashion models based on facial and skeletal features and emotions.

[0681] "Means for generating advice regarding optimal fashion items and styles and transmitting the advice to a user terminal together with additional suggestions based on emotions" refers to a technology for generating additional suggestions tailored to the user's emotional state along with fashion advice and transmitting the same from a server to a user terminal.

[0682] "Means for displaying generated advice on a user interface and display device" refers to functionality for visually presenting generated fashion advice and emotion-based suggestions on a user terminal and other display devices.

[0683] "Means using image processing technology and machine learning algorithms" refers to technology that uses image processing software and machine learning models to analyze image data and perform accurate feature extraction and emotion recognition.

[0684] A "fashion model database" refers to a data storage that accumulates information on various face shapes, body types, fashion items, and styles.

[0685] This system allows users to upload photos of their face and bone structure in a physical store, and provides fashion advice based on the user's emotions. The system consists of a user terminal, a server, an emotion engine, and a fashion model database.

[0686] Overall system configuration

[0687] User device functions

[0688] The user device is equipped with an application that allows users to take and select photos and upload them to a server. It also has the function of displaying fashion advice sent from the server and personalized recommendations based on emotions. Specifically, this corresponds to smart glasses or head-mounted displays installed in fitting rooms, etc.

[0689] Server Features

[0690] The server receives photos and facial expression data uploaded from the user's device and analyzes facial and skeletal features and emotions. The analysis uses image processing techniques (e.g., OpenCV), machine learning algorithms (e.g., TensorFlow / Keras), and an emotion engine (EmotionEngine). After analysis, the server compares the results with a database of fashion models and calculates advice based on the user's optimal fashion items, style, and emotions. The final advice is sent to the user's device.

[0691] Emotion Engine Functions

[0692] The Emotion Engine is a module that recognizes emotions from facial expression data uploaded by users. This engine uses machine learning algorithms to analyze the user's facial expressions and identify the emotion at that time. For example, if the Emotion Engine recognizes the emotion "joy" from the user's facial expression, it will recommend fashion items that match that emotion, including a relaxed style and cheerful color palette.

[0693] Fashion Model Database Features

[0694] This database stores information on fashion models with various facial shapes and skeletal features. The database includes information on each model's facial shape, body type, recommended fashion style, and item list. Specifically, if a model with a "round face and plump body" is registered, the style and items recommended by that model (e.g., A-line dresses, flared pants, etc.) will be recorded.

[0695] Fashion advice generation and transmission to user devices

[0696] After the server selects the optimal model, it generates specific fashion advice based on the model's data and the user's emotions. The advice includes recommended fashion items, styling details, and emotional suggestions. The generated advice is sent to the user's device and displayed on a user interface.

[0697] Explanation of program processing

[0698] 1. Hardware and Software Use

[0699] Camera (e.g. Logitech C920): Takes a photo of the user.

[0700] Emotion Engine: Analyze emotions using machine learning (e.g. TensorFlow / Keras).

[0701] Fashion Advice Server: A server that analyzes facial features and bone structure (e.g., built with Flask).

[0702] Fashion Model Database: Provides fashion model information based on facial and skeletal features.

[0703] Display device (e.g., smart glasses, HMD): displays advice to the user (e.g., Google Glass, HoloLens).

[0704] 2. Data processing or data calculation

[0705] Image Capture: Take a picture of the user with the camera and store it locally.

[0706] Image analysis: Sends the image to an analysis server to analyze the facial shape, bone structure, and emotion.

[0707] Database matching: The analysis results are compared with a fashion model database to select the most suitable model and style.

[0708] Advice generation: Generate personalized fashion advice based on the analysis results and information obtained from the model.

[0709] Adding specific examples

[0710] For example, if a user's facial expression captured by a "smart mirror" is "surprised," their face shape is "round," and their body type is "slim," the device will display the following message in real time:

[0711] "If you have a round face, a V-neck top is a great choice. A surprised look is great, so add some lighter colors to make it stand out even more."

[0712] Example prompts to input to the generative AI model

[0713] Generate optimal fashion advice for a user with a round face, slim figure, and surprised expression.

[0714] Create a system that provides real-time fashion advice based on facial shape and emotion.

[0715] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0716] Step 1:

[0717] The user takes a photo.

[0718] Input: The user takes a photo of their face and skeletal structure using smart glasses or a head-mounted display.

[0719] Data processing: The camera captures a photo of the user's face and overall appearance, and stores it locally as an image file.

[0720] Output: The saved image file (e.g. user_image.jpg).

[0721] Step 2:

[0722] The user terminal uploads the photo to the server.

[0723] Input: A saved image file.

[0724] Data processing: The user device sends the image file to the server.

[0725] Output: Image data sent to the server.

[0726] Step 3:

[0727] The server receives the image data and extracts facial and skeletal features and emotions.

[0728] Input: Image data sent to the server.

[0729] Data processing:

[0730] Step 1: The server uses image processing technology (e.g., OpenCV) to analyze the facial shape and skeletal features.

[0731] Step 2: The server uses a machine learning algorithm (e.g., TensorFlow / Keras) to identify the emotional state (e.g., "surprise," "joy," etc.) from the image.

[0732] Output: Extracted face shape, skeletal features, and emotion data.

[0733] Step 4:

[0734] The server compares the extracted features and emotions with a fashion model database and calculates the most suitable fashion.

[0735] Input: Extracted face shape, skeletal features, and emotion data.

[0736] Data processing:

[0737] Operation 1: The server accesses a fashion model database (e.g., FashionModelDatabase) and searches for the corresponding model data.

[0738] Action 2: Based on the collated data, the best fashion items and styles are selected for the user.

[0739] Output: Data about the best fashion items and styles.

[0740] Step 5:

[0741] The server generates advice and sends it to the user terminal.

[0742] Input: Data about optimal fashion items and styles, and sentiment data.

[0743] Data processing:

[0744] Action 1: The server generates fashion advice, which includes selected fashion items and styles, as well as additional suggestions based on the user's emotions.

[0745] Operation 2: The generated advice is sent to the user terminal.

[0746] Output: Fashion advice sent to the user's device.

[0747] Step 6:

[0748] The user terminal displays the generated advice.

[0749] Input: Fashion advice sent to user device.

[0750] Data processing: Displaying the generated advice on a user interface and display device (e.g., smart glasses, HMD).

[0751] Output: Fashion advice that the user can visually see.

[0752] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0753] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0754] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[0755] [Third embodiment]

[0756] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[0757] 5, the data processing system 310 includes the data processing device 12 and a headset type terminal 314. An example of the data processing device 12 is a server.

[0758] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0759] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[0760] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0761] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0762] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0763] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0764] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0765] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0766] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0767] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[0768] Overall system configuration

[0769] This invention is a system that allows users to upload photos of their face and bone structure and provides fashion advice based on those photos. The system is composed of a user terminal, a server, and a database of fashion models.

[0770] User device functions

[0771] The user device has an application that allows users to take and select photos and upload them to the server, and also has the function of displaying fashion advice sent from the server.

[0772] Example: A user launches an app, uses the camera to take a photo of their face and body, and then uploads the photo to a server through the app's interface.

[0773] Server Features

[0774] The server receives photos uploaded from the user's device and analyzes facial and skeletal features using image processing technology and machine learning algorithms. After analysis, the photos are compared with a database of fashion models to calculate the most suitable fashion items and style for the user. Final advice is then sent to the user's device.

[0775] Example: The server receives the uploaded photo and applies a facial recognition algorithm to extract facial shape and skeletal features. The extracted data is compared with a database to select the most suitable fashion model. For example, if the user's characteristics are "long face, slim figure," the server will search the database for a model with the closest characteristics.

[0776] Fashion Model Database Features

[0777] This database stores information on fashion models with various facial shapes and bone structures. The database includes each model's facial shape, body shape, recommended fashion style, and item list.

[0778] Example: If a model with a "round face and plump figure" is registered in the database, the style and items recommended by that model (e.g., A-line dresses, flared pants, etc.) will be recorded.

[0779] Fashion advice generation and transmission to user devices

[0780] After the server selects the optimal model, it generates specific fashion advice based on the model's data. The advice includes recommended fashion items and styling details. The advice is then sent to the user's device and displayed on a user interface.

[0781] Example: Based on the user's characteristics, specific styling advice is generated, such as "V-neck tops and high-waisted pants look good on you." This advice is sent to the user's device and displayed within the app.

[0782] Use Case Details

[0783] For example, if a user has a "long face and slim figure," the server extracts the corresponding features from the database and determines that "Model A" is the best fit. Model A recommends that "V-neck tops and high-waisted pants look good on you," and this styling advice is provided to the user. The user can check this advice within the app and use it as a reference for shopping and coordinating outfits.

[0784] The system allows users to receive accurate fashion advice based on their facial and bone structure, making it easy to find the style that best suits them.

[0785] The processing flow will be explained below.

[0786] Step 1:

[0787] The user launches the smartphone app and takes a photo of their face and whole body, or selects an existing photo.

[0788] Step 2:

[0789] Users upload photos to the server through the app's interface, and the photo data is transmitted over secure communication.

[0790] Step 3:

[0791] The server receives photo data uploaded by the user.

[0792] Step 4:

[0793] The server passes the received photo data to the image processing module for analysis, where the face detection and feature extraction process begins.

[0794] Step 5:

[0795] The server's image processing module extracts facial shape and skeletal features, such as facial contours, eye, nose, and mouth positions, and skeletal proportions.

[0796] Step 6:

[0797] The server then compares the extracted feature data with a database of fashion models, which contains data on models with various facial and body shapes.

[0798] Step 7:

[0799] The server searches the database for a model that best matches the user's characteristics and calculates the optimal fashion style based on that model.

[0800] Step 8:

[0801] The server generates advice on optimal fashion items and styles, including specific clothing types and styling suggestions.

[0802] Step 9:

[0803] The server transmits the generated fashion advice to the user's terminal.

[0804] Step 10:

[0805] The user's device receives the advice data sent from the server and displays it within the application, allowing the user to check it and use it to help them choose their own fashion.

[0806] Example 1

[0807] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[0808] Conventional fashion advice systems often have difficulty providing personalized advice that takes into account the user's facial and body shapes. Furthermore, they have faced the problem of requiring time and effort for users to find the fashion items and styling that best suit them. This makes it difficult for users to easily find the style that best suits them.

[0809] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0810] In this invention, the server includes means for allowing users to upload photos of their face and skeletal structure, means for receiving the uploaded photos and extracting facial and skeletal features, means for comparing the extracted features with a fashion model database to calculate the most suitable fashion for the user, means for generating advice on the most suitable fashion items and style and sending it to the user terminal, means for displaying the generated advice on a user interface, and means for performing a facial recognition algorithm, a machine learning algorithm, and database comparison. This allows users to receive accurate fashion advice based on their facial and skeletal features, making it easy to find the style that best suits them.

[0811] "User Device" means a device equipped with an application that allows a user to take and select photos and upload them to the server, and that has the functionality to display fashion advice sent from the server.

[0812] The "server" is a central processing unit that receives photos uploaded from the user's terminal, analyzes facial and skeletal features, compares the analysis results with a database of fashion models to calculate the most suitable fashion items and style for the user, and sends the generated advice to the user's terminal.

[0813] A "fashion model database" is a database that stores information on fashion models with various facial shapes and skeletal features, and is a data storage that includes each model's facial shape, body shape, recommended fashion style, item list, etc.

[0814] A "facial recognition algorithm" is an algorithm used to analyze the shape and features of a face from an uploaded photo, and is a technology that uses image processing libraries such as OpenCV and Dlib to detect facial contours and feature points.

[0815] A "machine learning algorithm" is an algorithm that performs classification and prediction based on extracted facial and skeletal feature data, and is a technology that analyzes feature data using methods such as Random Forest and SVM (Support Vector Machine).

[0816] "Fashion items" are specific clothing, accessories, and other fashion products that are suggested to a user and are recommended based on the user's facial and skeletal features.

[0817] A "prompt" is an instruction entered into a generative AI model to output appropriate fashion advice, and is text input to obtain optimal output based on the user's characteristics and requests.

[0818] MODE FOR CARRYING OUT THE INVENTION

[0819] Overall system configuration

[0820] This invention is a system that allows users to upload photos of their face and bone structure and provides fashion advice based on those photos. The system is composed of a user terminal, a server, and a database of fashion models.

[0821] User device functions

[0822] The user device has an application that allows users to take and select photos and upload them to the server. It also has a function to display fashion advice sent from the server. Users launch the app, take photos of their face and body using the camera, and then upload the photos to the server through the app interface.

[0823] Examples:

[0824] The user launches the app on their smartphone, takes a photo of their face and whole body, and then clicks the app's upload button to send the photo to the server.

[0825] Server Features

[0826] The server receives photos uploaded from the user's device and analyzes facial and skeletal features. This analysis uses image processing technology and machine learning algorithms. Specifically, OpenCV and Dlib are used to detect facial shape and feature points, and the data is analyzed using machine learning algorithms such as Random Forest and SVM. After analysis, the results are compared with a fashion model database to calculate the optimal fashion items and style for the user. Final advice is then sent to the user's device.

[0827] Examples:

[0828] The server receives the photo sent by the user and performs facial recognition using OpenCV. It extracts facial feature points using Dlib and then applies the Random Forest algorithm to identify the user's facial and skeletal characteristics. It then compares the results with a fashion model database to select a model that most closely matches the "long face, slim body shape" description and calculates the model's recommended fashion style.

[0829] Fashion Model Database Features

[0830] This database stores information on fashion models with various facial shapes and bone structures. The database includes each model's facial shape, body shape, recommended fashion style, and item list.

[0831] Examples:

[0832] If a model with a "round face and plump figure" is registered in the database, the style and items that model recommends (e.g., A-line dresses, flared pants, etc.) are recorded.

[0833] Fashion advice generation and transmission to user devices

[0834] After the server selects the optimal model, it generates specific fashion advice based on the model's data. The advice includes recommended fashion items and styling details. The advice is sent to the user's device and displayed on a user interface.

[0835] Examples:

[0836] Based on the user's characteristics, specific styling advice is generated, such as "V-neck tops and high-waisted pants look good on you." This advice is sent to the user's device and displayed within the app.

[0837] Prompt Sentence Examples

[0838] A typical example of a prompt to be input to a generative AI model might be:

[0839] "Based on photos of the user's face and body, please suggest the most suitable fashion items and styles. The user's characteristics are as follows: 'Long face, slim body'. Please output the most suitable fashion advice for this."

[0840] Using this prompt, the generative AI model outputs content that provides appropriate fashion advice.

[0841] The system allows users to receive accurate fashion advice based on their facial and bone structure, making it easy to find the style that best suits them.

[0842] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0843] Step 1:

[0844] Receiving and storing photos on the server

[0845] The server receives photos uploaded from the user's device. This input is a face and full-body photo sent by the user through the app. The server saves the received photos in temporary storage. As a concrete example, the server receives image data through an HTTP request and saves the data in local storage.

[0846] Input: Face and full-body photo data sent from the user device.

[0847] Data processing: Analyzing HTTP requests and saving image data.

[0848] Output: Saved image file.

[0849] Step 2:

[0850] Server-based face recognition and skeletal feature extraction

[0851] The server analyzes the stored photos to extract facial shape and skeletal features. This step uses image processing technologies such as OpenCV and Dlib. First, a facial recognition algorithm is applied, and then Dlib is used to extract the coordinates of facial feature points (eyes, nose, mouth, etc.).

[0852] Input: Saved face and full-body image files.

[0853] Data processing: Face recognition using OpenCV and feature point extraction using Dlib.

[0854] Output: Facial shape and skeletal feature coordinate data.

[0855] Step 3:

[0856] Application of machine learning algorithms by the server

[0857] The server inputs the extracted facial and skeletal feature data into a machine learning algorithm to classify the user's face shape and body shape. The algorithms used include Random Forest and SVM. In this step, the feature data is input into the machine learning model as a numerical vector to obtain the classification results.

[0858] Input: Coordinate data of facial shape and skeletal features.

[0859] Data processing: Vectorizing feature data and inputting it into a machine learning model.

[0860] Output: Classified face shape and body shape data.

[0861] Step 4:

[0862] Database verification by the server

[0863] The server compares the classified face shape and body shape data with a fashion model database, which contains each model's face shape, body shape, recommended fashion style, and items. In this step, the server searches for the most suitable model and obtains the model's data.

[0864] Input: Classified face shape and body shape data.

[0865] Data processing: Search the fashion model database and select the most suitable model.

[0866] Output: Data of the best fashion model.

[0867] Step 5:

[0868] Server-based fashion advice generation

[0869] The server generates specific fashion advice based on the data of the selected fashion model. Using the generative AI model, it generates text that suggests the most suitable fashion items and styles for the user. For example, it might generate advice such as "V-neck tops and high-waisted pants look good on you."

[0870] Input: Data of the optimal fashion model.

[0871] Data processing: Generate text advice using a generative AI model.

[0872] Output: Text data of fashion advice.

[0873] Step 6:

[0874] Server sends advice and displays it on the user's device

[0875] The server sends the generated fashion advice to the user's device, which then displays the received advice within the application. Specifically, the server sends the advice as an HTTP response, and the user's device analyzes it and displays it on the interface.

[0876] Input: Text data of fashion advice.

[0877] Data processing: Sending advice and analyzing received data.

[0878] Output: User interface with advice displayed.

[0879] (Application example 1)

[0880] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[0881] Fashion-conscious users face challenges in easily finding the styles and items that best suit them. In particular, there are few methods for quickly providing personalized fashion advice based on a user's face shape and bone structure, making it time-consuming and labor-intensive to select a style that suits them. Furthermore, there is no way to immediately purchase products based on the advice provided, which can disrupt the user's shopping experience. To address these issues, a system is needed that can quickly and accurately analyze a user's characteristics, provide optimal fashion advice, and enable the user to purchase products based on the advice.

[0882] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0883] In this invention, the server includes means for allowing a user to upload images of their face and bone structure, means for receiving the uploaded images and extracting facial and bone structure features, means for comparing the extracted features with a fashion model database to calculate the most suitable fashion for the user, means for generating advice on the most suitable fashion items and style and transmitting it to the user terminal, means for displaying the generated advice on a user interface, and means for allowing the user to purchase products based on the advice through an application. This allows the user to receive the most suitable fashion advice based on their own features and instantly purchase products based on the advice.

[0884] A "user terminal" is a device that allows a user to take photos, upload images to a server through an application, and receive fashion advice.

[0885] The "server" is a computer system that receives images sent from a user terminal, analyzes them, generates fashion advice, and sends it to the user terminal.

[0886] "Image processing technology" is a technology for analyzing images taken by a user and extracting facial and skeletal features.

[0887] "Machine learning algorithms" are artificial intelligence technologies that compare facial and skeletal features with a database to generate optimal fashion advice for users.

[0888] A "fashion model database" is a collection of information that stores data on models based on various face shapes and body shapes, and is used to provide fashion advice based on that data.

[0889] The "user interface" refers to a screen or operating means for displaying the generated fashion advice to the user.

[0890] A "prompt sentence" is text data that uses a generative AI model to interactively analyze the user's image and features.

[0891] The "shopping function" is a function that allows a user to directly purchase items shown in the advice based on the generated fashion advice.

[0892] This invention is a system that allows users to upload images of their face and bone structure and provides fashion advice based on those images. The system consists of a user terminal, a server, and a database of fashion models.

[0893] User device functions

[0894] The user terminal has a function that allows the user to take pictures and upload them to the server through the application, and also provides an interface that displays fashion advice sent from the server and allows the user to purchase products based on that advice.

[0895] For example, a user launches the app and takes a picture of their face and whole body using their camera. They then upload the image to a server through the app's interface. After uploading, the user waits for a response from the server and can view the resulting advice on the screen. Based on this advice, a link or button to purchase the product is provided, allowing the user to continue shopping.

[0896] Server Features

[0897] The server receives the images sent from the user terminal and extracts facial and skeletal features using image processing techniques and machine learning algorithms.

[0898] Specifically, the server does the following:

[0899] 1. Image reception: Has an API endpoint for receiving image files sent from the user device.

[0900] 2. Feature extraction: The received image is analyzed and facial and skeletal features are extracted using a facial recognition algorithm, such as the Python face_recognition library. Numerical data such as facial shape and skeletal proportions are obtained.

[0901] 3. Database Matching: The extracted features are matched against a database of fashion models, which contains feature data for each model and fashion styles based on those features.

[0902] 4. Advice Generation: Select the best model and generate styling advice based on that model. This advice is crafted in detail using the generative AI model and prompt text.

[0903] 5. Sending advice: Install an API to send the generated advice to the user's device.

[0904] Fashion Model Database Features

[0905] The fashion model database stores information on models with various facial shapes and bone structures. The database includes each model's facial shape, body shape, recommended fashion style, and item list.

[0906] For example, if a model with a "round face and plump figure" is registered in the database, the style and clothing that model recommends (e.g., A-line dresses, flared pants, etc.) are recorded. This allows the server to provide advice that best suits the user's characteristics.

[0907] Fashion advice generation and transmission to user devices

[0908] After the server selects the optimal model, it generates specific fashion advice based on the model's data. The advice includes recommended fashion items and styling details. The advice is sent to the user's device and displayed on a user interface.

[0909] For example, styling advice such as "V-neck tops and high-waisted pants look good on you" is generated based on the user's characteristics. This advice is sent to the user's device and displayed within the app. The user can also purchase items directly within the app.

[0910] Prompt Sentence Examples

[0911] "Analyze the user's photo and extract facial shape and bone structure features. Based on those features, provide the most suitable fashion items and styling advice."

[0912] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0913] Step 1:

[0914] The user launches the smartphone app and takes a photo of their face and whole body.

[0915] In this step, the user takes a photo using the app's camera function and the photo is saved in the smartphone. The input is the original photo taken by the user, and the output is the photo data saved in the smartphone.

[0916] Step 2:

[0917] The user terminal uploads the photograph to the server.

[0918] In this step, when a user clicks the upload button in the app, the photo file is sent to the server through the HTTP request process within the app. The input is the photo data stored on the smartphone, and the output is the data sent to the server.

[0919] Step 3:

[0920] The server receives the photo data and uses image processing techniques and machine learning algorithms to extract facial and skeletal features.

[0921] Specifically, the server analyzes the photo using Python's face_recognition library and extracts the facial shape and skeletal features as numerical data. The input is the photo data sent to the server, and the output is the extracted facial and skeletal feature data.

[0922] Step 4:

[0923] Based on the extracted feature data, the server compares it with a database of fashion models and calculates the most suitable fashion for the user.

[0924] The server uses the extracted feature data to compare it with the features of each model stored in the database and selects the optimal model. The input is the extracted feature data and a database of fashion models, and the output is the optimal model data and styling advice.

[0925] Step 5:

[0926] The server uses the generated AI model and prompts to generate specific fashion advice.

[0927] In this step, the server inputs prompts into the generative AI model to generate specific fashion item and style advice based on the user's characteristics. The input is the optimal model's data and prompts, and the output is specific fashion advice.

[0928] Step 6:

[0929] The server transmits the generated advice to the user terminal.

[0930] The server sends the generated advice to the user terminal as an HTTP response. The input is the specific fashion advice, and the output is the advice data sent to the user terminal.

[0931] Step 7:

[0932] The user terminal displays the received advice on a user interface, and the user purchases the product based on the advice.

[0933] In this step, the user can review the advice displayed within the app and purchase the product by clicking a purchase link or button. The input is the advice data received from the server, and the output is the advice and purchase procedure displayed on the user's display screen.

[0934] Prompt Sentence Examples

[0935] "Analyze the user's photo and extract facial shape and bone structure features. Based on those features, provide the most suitable fashion items and styling advice."

[0936] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0937] Overall system configuration

[0938] This invention is a system that allows users to upload photos of their face and bone structure, and provides fashion advice based on the user's emotions. The system consists of a user terminal, a server, an emotion engine, and a database of fashion models.

[0939] User device functions

[0940] The user device is equipped with an application that allows users to take and select photos and upload them to the server, and also has the function of displaying fashion advice and personalized recommendation advice based on emotions sent from the server.

[0941] Example: A user launches an app, uses the camera to take a photo of their face and body, captures facial expressions to recognize their current emotion, and then uploads the photo to a server through the app interface.

[0942] Server Features

[0943] The server receives photos and facial expression data uploaded from the user's device and analyzes facial and skeletal features and emotions. The analysis uses image processing technology, machine learning algorithms, and an emotion engine. After analysis, the server compares the results with a database of fashion models and calculates advice based on the user's optimal fashion items, style, and emotions. The final advice is then sent to the user's device.

[0944] Example: The server receives the uploaded photo and facial expression data, and applies facial recognition and emotion recognition algorithms to extract facial shape, skeletal features, and the current emotional state (e.g., "joy" or "sadness"). The extracted data is compared with a database to select the most suitable fashion model.

[0945] Emotion Engine Functions

[0946] The emotion engine is a module for recognizing emotions from facial expression data uploaded by users. This engine uses machine learning algorithms to analyze the user's facial expressions and identify the emotion at that time.

[0947] Example: If the emotion engine recognizes the emotion "joy" from a user's facial expression, it will recommend fashion items that match that emotion, including relaxed styles and cheerful color palettes.

[0948] Fashion Model Database Features

[0949] This database stores information on fashion models with various facial shapes and bone structures. The database includes each model's facial shape, body shape, recommended fashion style, and item list.

[0950] Example: If a model with a "round face and plump figure" is registered in the database, the style and items recommended by that model (e.g., A-line dresses, flared pants, etc.) will be recorded.

[0951] Fashion advice generation and transmission to user devices

[0952] After the server selects the optimal model, it generates specific fashion advice based on the model's data and the user's emotions. The advice includes recommended fashion items, styling details, and emotional suggestions. The generated advice is sent to the user's device and displayed on a user interface.

[0953] Example: Based on the user's characteristics and emotions, specific styling advice such as "V-neck tops and high-waisted pants look good on you" is provided, along with additional emotional advice such as "Bright colors are recommended for today's mood." This advice is sent to the user's device and displayed within the app.

[0954] Use Case Details

[0955] For example, if a user has a "long face, slim figure," and is currently feeling "happy," the server will extract the corresponding features and emotions from the database and determine that "Model A" is the best fit. Model A recommends that "V-neck tops and high-waisted pants look good on you," so in addition to this styling advice, it will provide the user with emotion-based advice such as "bright-colored items are perfect for your mood today." Users can check these advice within the app and use them as reference for shopping and outfits.

[0956] The system allows users to receive accurate fashion advice based on their facial and skeletal features and emotions, making it easy to find the style that best suits them.

[0957] The processing flow will be explained below.

[0958] Step 1:

[0959] The user launches the smartphone app and takes a photo of their face, their whole body, and their facial expression, or selects an existing photo.

[0960] Step 2:

[0961] Users upload photo data and facial expression data to the server through the app interface, and the data is transmitted over secure communication.

[0962] Step 3:

[0963] The server receives the photo data and facial expression data uploaded by the user.

[0964] Step 4:

[0965] The server then passes the received photo data to the image processing module for analysis, where the process of recognizing facial and skeletal features and facial expressions begins.

[0966] Step 5:

[0967] The server's image processing module extracts facial shape and skeletal features, such as facial contours, eye, nose, and mouth positions, and skeletal proportions.

[0968] Step 6:

[0969] Furthermore, an emotion engine analyzes the facial expression data to identify the user's current emotion, such as "happiness," "sadness," or "surprise."

[0970] Step 7:

[0971] The server compares the extracted feature data and emotion data with a database of fashion models, which contains data on models with various facial and body shapes.

[0972] Step 8:

[0973] The server searches the database for a model that best matches the user's facial and skeletal features and emotional data, and calculates the optimal fashion style based on that model.

[0974] Step 9:

[0975] The server generates advice about the calculated fashion items and styles, including recommended clothing types, styling details, and additional suggestions depending on the emotion.

[0976] Step 10:

[0977] The server sends the generated fashion advice to the user's device, including colors and styling that match the user's emotions.

[0978] Step 11:

[0979] The user's device receives the advice data sent from the server and displays it within the application, allowing the user to check it and use it to help them choose their own fashion.

[0980] Step 12:

[0981] Based on the advice provided, the user purchases the recommended fashion items or puts together an outfit.

[0982] Example 2

[0983] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[0984] Conventional fashion advice systems only consider the user's facial and skeletal features, making it difficult to provide advice that reflects the user's current emotions. Furthermore, advice that does not consider emotions makes it difficult to select the optimal fashion that matches the user's feelings. Therefore, there is a need for a method to support users in selecting fashion that suits their daily mood and the occasion.

[0985] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0986] In this invention, the server includes a means for allowing a user to upload a photograph of their face and skeletal structure, a means for receiving the uploaded photograph and extracting facial and skeletal features, and a means for analyzing the user's emotions using an emotion engine, thereby enabling optimal fashion advice to be provided to the user based on the extracted features and analyzed emotions.

[0987] "User" refers to any person who uses the system to upload their own photos and receive fashion advice.

[0988] "Server" refers to a computer system that receives data uploaded by users and performs analysis and advice generation.

[0989] "Photo" refers to image data including a user's face and bone structure that is uploaded by the user.

[0990] "Facial and skeletal features" refers to data being analyzed that indicates the user's physical characteristics, such as facial shape and skeletal structure.

[0991] "Emotion engine" refers to a module that analyzes and identifies emotions from a user's facial expression data.

[0992] A "fashion model database" refers to an information repository that stores data on each model based on their face shape and body type.

[0993] "Means for generating advice" refers to the process of creating recommendations for the most suitable fashion items and styles for the user based on the analyzed features and emotional data.

[0994] "User Terminal" refers to the device used by a User to upload photos and receive and view advice.

[0995] "User interface" refers to the screen and interaction mechanism that displays advice generated on the user's terminal and allows the user to operate it.

[0996] "Image processing techniques" refers to techniques used to extract facial and skeletal features from photographs.

[0997] "Machine learning algorithms" refer to computational techniques used for data analysis and emotion recognition.

[0998] The system of this invention allows users to upload photos of their face and bone structure, and provides appropriate fashion advice based on the user's emotions. The system is composed of a user terminal, a server, an emotion engine, and a database of fashion models.

[0999] User device functions

[1000] The user device is equipped with an application that allows users to take and select photos and upload them to the server, and also has the function of displaying fashion advice and personalized recommendation advice based on emotions sent from the server.

[1001] Specifically, the user launches the app and takes a photo of their face and whole body using the camera. Furthermore, facial expressions are captured to recognize the current emotion, and the captured images are uploaded to a server through the app's interface. The hardware used is a mobile device such as a smartphone or tablet, and the app is developed based on a general-purpose application such as a "movie viewer for mobile devices."

[1002] Server Features

[1003] The server receives photos and facial expression data uploaded from the user's device and analyzes facial and skeletal features and emotions. Image processing technology, image processing libraries such as "OpenCV," and machine learning algorithms such as "DeepFace" are used for the analysis. The server then compares the analysis results with a database of fashion models to calculate advice based on the user's optimal fashion items, style, and emotions. The final advice is then sent to the user's device.

[1004] Specifically, the server receives a photo and facial expression data from the user, extracts facial shape and skeletal features, and simultaneously recognizes the user's emotional state (e.g., "joy" or "sadness"). The extracted data is compared with a database of fashion models, and suggestions for optimal fashion items, styling details, and emotions are generated. This data is then sent to the user's device, where the user can view it within the app.

[1005] Emotion Engine Functions

[1006] The emotion engine is a module for recognizing emotions from facial expression data uploaded by users. This engine uses machine learning algorithms to analyze the user's facial expressions and recognize specific emotions.

[1007] For example, if the emotion engine recognizes the emotion of "joy" from the user's facial expression, it will recommend fashion items that match that emotion, including a relaxed style and cheerful color palette. The machine learning algorithms used can be "TensorFlow" or "Keras."

[1008] Fashion Model Database Features

[1009] The fashion model database stores information on fashion models with various facial shapes and skeletal features. The database includes each model's facial shape, body shape, recommended fashion style, and item list.

[1010] Specifically, if a model with a "round face and plump figure" is registered in the database, the style and items recommended by that model (e.g., A-line dress, flared pants, etc.) are recorded.

[1011] Fashion advice generation and transmission to user devices

[1012] After the server selects the optimal model, it generates specific fashion advice based on the model's data and the user's emotions. The advice includes recommended fashion items, styling details, and emotional suggestions. The generated advice is sent to the user's device and displayed on a user interface.

[1013] As a specific example, based on the user's characteristics and emotions, in addition to specific styling advice such as "V-neck tops and high-waisted pants look good on you," additional emotional advice such as "bright colors are recommended for today's mood" is provided.

[1014] Prompt Sentence Examples

[1015] "You will create a program that allows users to upload photos of their face and bone structure, recognizes emotions, and provides optimal fashion advice. Using image processing technology and machine learning algorithms, you will design a system that analyzes facial and bone structure features and emotions, and compares them with a database of fashion models to generate advice."

[1016] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1017] Step 1:

[1018] Users take or select a photo of their face and bone structure and upload it to a server through the app.

[1019] Specifically, the user launches the app and taps the "Take a Photo" button. The user then uses the camera to take a photo of their face and whole body, and taps the "Upload" button afterward. The input is the user's photo data, and the output is the photo data to be sent to the server.

[1020] Step 2:

[1021] The server receives the photo data uploaded by the user and temporarily stores it.

[1022] Specifically, the server receives photo data via an HTTP request and temporarily stores it in a database. The input is photo data from the user's device, and the output is the image data stored on the server.

[1023] Step 3:

[1024] The server analyzes the stored photographic data and extracts facial and skeletal features.

[1025] Specifically, the server uses OpenCV to perform facial recognition and extract skeletal features. It applies image processing algorithms to detect facial contours and skeletal points. The input is temporarily saved image data, and the output is facial and skeletal feature data.

[1026] Step 4:

[1027] The emotion engine analyzes facial expression data and recognizes specific emotions.

[1028] Specifically, facial expression data is fed into a machine learning algorithm such as "DeepFace" to recognize emotions (e.g., "happiness" or "sadness"). The input is facial expression data, and the output is emotional data.

[1029] Step 5:

[1030] The server compares the extracted facial and skeletal feature data and emotion data with a database of fashion models.

[1031] Specifically, the server issues a query to the fashion model database to retrieve data on models with similar features and emotions. The input is feature data and emotion data, and the output is the optimal model data.

[1032] Step 6:

[1033] The server generates specific fashion advice based on the acquired model data and the user's emotional data.

[1034] Specifically, the server analyzes the recommended fashion items and styling details based on the acquired model data, and generates additional suggestions based on the user's emotions. The inputs are model data and emotion data, and the output is generated advice data.

[1035] Step 7:

[1036] The generated advice data is sent to the user terminal, and the user confirms the advice through the app.

[1037] Specifically, the server sends the generated advice data to the user device via the endpoint. The user device receives the advice data from the endpoint and displays it on the app interface. The input is the generated advice data, and the output is the advice displayed on the user device.

[1038] (Application example 2)

[1039] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1040] Conventional fashion advice systems suggest styles based on the user's facial and skeletal features, but are unable to consider the user's emotional state. This makes it difficult to provide more personalized advice tailored to the user's mood. It is also difficult to provide real-time fashion advice in physical stores. Therefore, there is a need for the development of a system that can consider the user's emotions and provide optimal fashion advice in real time.

[1041] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1042] In this invention, the server includes means for allowing a user to upload a photograph of their face and skeletal structure, means for receiving the uploaded photograph and extracting facial and skeletal features and emotions, means for comparing the extracted features and emotions with a fashion model database to calculate the most suitable fashion for the user, means for generating advice on the most suitable fashion items and style and sending it to the user terminal together with additional suggestions based on the emotions, and means for displaying the generated advice on a user interface and a display device, thereby enabling the provision of personalized fashion advice that takes the user's emotions into consideration.

[1043] "Means for enabling users to upload photographs of their own face and bones" refers to a function for taking or selecting photographs of the face and bones using the user terminal and sending them to the server.

[1044] "Means for the server to receive uploaded photos and extract facial and skeletal features and emotions" refers to technology in which the server analyzes image data sent from the user terminal and identifies the facial shape, skeletal features, and current emotional state.

[1045] "Means for comparing the extracted features and emotions with a database of fashion models and calculating the most suitable fashion for the user" refers to a function that recommends the most suitable fashion style and items from a database of fashion models based on facial and skeletal features and emotions.

[1046] "Means for generating advice regarding optimal fashion items and styles and transmitting the advice to a user terminal together with additional suggestions based on emotions" refers to a technology for generating additional suggestions tailored to the user's emotional state along with fashion advice and transmitting the same from a server to a user terminal.

[1047] "Means for displaying generated advice on a user interface and display device" refers to functionality for visually presenting generated fashion advice and emotion-based suggestions on a user terminal and other display devices.

[1048] "Means using image processing technology and machine learning algorithms" refers to technology that uses image processing software and machine learning models to analyze image data and perform accurate feature extraction and emotion recognition.

[1049] A "fashion model database" refers to a data storage that accumulates information on various face shapes, body types, fashion items, and styles.

[1050] This system allows users to upload photos of their face and bone structure in a physical store, and provides fashion advice based on the user's emotions. The system consists of a user terminal, a server, an emotion engine, and a fashion model database.

[1051] Overall system configuration

[1052] User device functions

[1053] The user device is equipped with an application that allows users to take and select photos and upload them to a server. It also has the function of displaying fashion advice sent from the server and personalized recommendations based on emotions. Specifically, this corresponds to smart glasses or head-mounted displays installed in fitting rooms, etc.

[1054] Server Features

[1055] The server receives photos and facial expression data uploaded from the user's device and analyzes facial and skeletal features and emotions. The analysis uses image processing techniques (e.g., OpenCV), machine learning algorithms (e.g., TensorFlow / Keras), and an emotion engine (EmotionEngine). After analysis, the server compares the results with a database of fashion models and calculates advice based on the user's optimal fashion items, style, and emotions. The final advice is sent to the user's device.

[1056] Emotion Engine Functions

[1057] The Emotion Engine is a module that recognizes emotions from facial expression data uploaded by users. This engine uses machine learning algorithms to analyze the user's facial expressions and identify the emotion at that time. For example, if the Emotion Engine recognizes the emotion "joy" from the user's facial expression, it will recommend fashion items that match that emotion, including a relaxed style and cheerful color palette.

[1058] Fashion Model Database Features

[1059] This database stores information on fashion models with various facial shapes and skeletal features. The database includes information on each model's facial shape, body type, recommended fashion style, and item list. Specifically, if a model with a "round face and plump body" is registered, the style and items recommended by that model (e.g., A-line dresses, flared pants, etc.) will be recorded.

[1060] Fashion advice generation and transmission to user devices

[1061] After the server selects the optimal model, it generates specific fashion advice based on the model's data and the user's emotions. The advice includes recommended fashion items, styling details, and emotional suggestions. The generated advice is sent to the user's device and displayed on a user interface.

[1062] Explanation of program processing

[1063] 1. Hardware and Software Use

[1064] Camera (e.g. Logitech C920): Takes a photo of the user.

[1065] Emotion Engine: Analyze emotions using machine learning (e.g. TensorFlow / Keras).

[1066] Fashion Advice Server: A server that analyzes facial features and bone structure (e.g., built with Flask).

[1067] Fashion Model Database: Provides fashion model information based on facial and skeletal features.

[1068] Display device (e.g., smart glasses, HMD): displays advice to the user (e.g., Google Glass, HoloLens).

[1069] 2. Data processing or data calculation

[1070] Image Capture: Take a picture of the user with the camera and store it locally.

[1071] Image analysis: Sends the image to an analysis server to analyze the facial shape, bone structure, and emotion.

[1072] Database matching: The analysis results are compared with a fashion model database to select the most suitable model and style.

[1073] Advice generation: Generate personalized fashion advice based on the analysis results and information obtained from the model.

[1074] Adding specific examples

[1075] For example, if a user's facial expression captured by a "smart mirror" is "surprised," their face shape is "round," and their body type is "slim," the device will display the following message in real time:

[1076] "If you have a round face, a V-neck top is a great choice. A surprised look is great, so add some lighter colors to make it stand out even more."

[1077] Example prompts to input to the generative AI model

[1078] Generate optimal fashion advice for a user with a round face, slim figure, and surprised expression.

[1079] Create a system that provides real-time fashion advice based on facial shape and emotion.

[1080] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1081] Step 1:

[1082] The user takes a photo.

[1083] Input: The user takes a photo of their face and skeletal structure using smart glasses or a head-mounted display.

[1084] Data processing: The camera captures a photo of the user's face and overall appearance, and stores it locally as an image file.

[1085] Output: The saved image file (e.g. user_image.jpg).

[1086] Step 2:

[1087] The user terminal uploads the photo to the server.

[1088] Input: A saved image file.

[1089] Data processing: The user device sends the image file to the server.

[1090] Output: Image data sent to the server.

[1091] Step 3:

[1092] The server receives the image data and extracts facial and skeletal features and emotions.

[1093] Input: Image data sent to the server.

[1094] Data processing:

[1095] Step 1: The server uses image processing technology (e.g., OpenCV) to analyze the facial shape and skeletal features.

[1096] Step 2: The server uses a machine learning algorithm (e.g., TensorFlow / Keras) to identify the emotional state (e.g., "surprise," "joy," etc.) from the image.

[1097] Output: Extracted face shape, skeletal features, and emotion data.

[1098] Step 4:

[1099] The server compares the extracted features and emotions with a fashion model database and calculates the most suitable fashion.

[1100] Input: Extracted face shape, skeletal features, and emotion data.

[1101] Data processing:

[1102] Operation 1: The server accesses a fashion model database (e.g., FashionModelDatabase) and searches for the corresponding model data.

[1103] Action 2: Based on the collated data, the best fashion items and styles are selected for the user.

[1104] Output: Data about the best fashion items and styles.

[1105] Step 5:

[1106] The server generates advice and sends it to the user terminal.

[1107] Input: Data about optimal fashion items and styles, and sentiment data.

[1108] Data processing:

[1109] Action 1: The server generates fashion advice, which includes selected fashion items and styles, as well as additional suggestions based on the user's emotions.

[1110] Operation 2: The generated advice is sent to the user terminal.

[1111] Output: Fashion advice sent to the user's device.

[1112] Step 6:

[1113] The user terminal displays the generated advice.

[1114] Input: Fashion advice sent to user device.

[1115] Data processing: Displaying the generated advice on a user interface and display device (e.g., smart glasses, HMD).

[1116] Output: Fashion advice that the user can visually see.

[1117] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1118] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1119] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[1120] [Fourth embodiment]

[1121] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[1122] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1123] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1124] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[1125] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1126] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1127] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1128] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[1129] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1130] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1131] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1132] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1133] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1134] Overall system configuration

[1135] This invention is a system that allows users to upload photos of their face and bone structure and provides fashion advice based on those photos. The system is composed of a user terminal, a server, and a database of fashion models.

[1136] User device functions

[1137] The user device has an application that allows users to take and select photos and upload them to the server, and also has the function of displaying fashion advice sent from the server.

[1138] Example: A user launches an app, uses the camera to take a photo of their face and body, and then uploads the photo to a server through the app's interface.

[1139] Server Features

[1140] The server receives photos uploaded from the user's device and analyzes facial and skeletal features using image processing technology and machine learning algorithms. After analysis, the photos are compared with a database of fashion models to calculate the most suitable fashion items and style for the user. Final advice is then sent to the user's device.

[1141] Example: The server receives the uploaded photo and applies a facial recognition algorithm to extract facial shape and skeletal features. The extracted data is compared with a database to select the most suitable fashion model. For example, if the user's characteristics are "long face, slim figure," the server will search the database for a model with the closest characteristics.

[1142] Fashion Model Database Features

[1143] This database stores information on fashion models with various facial shapes and bone structures. The database includes each model's facial shape, body shape, recommended fashion style, and item list.

[1144] Example: If a model with a "round face and plump figure" is registered in the database, the style and items recommended by that model (e.g., A-line dresses, flared pants, etc.) will be recorded.

[1145] Fashion advice generation and transmission to user devices

[1146] After the server selects the optimal model, it generates specific fashion advice based on the model's data. The advice includes recommended fashion items and styling details. The advice is then sent to the user's device and displayed on a user interface.

[1147] Example: Based on the user's characteristics, specific styling advice is generated, such as "V-neck tops and high-waisted pants look good on you." This advice is sent to the user's device and displayed within the app.

[1148] Use Case Details

[1149] For example, if a user has a "long face and slim figure," the server extracts the corresponding features from the database and determines that "Model A" is the best fit. Model A recommends that "V-neck tops and high-waisted pants look good on you," and this styling advice is provided to the user. The user can check this advice within the app and use it as a reference for shopping and coordinating outfits.

[1150] The system allows users to receive accurate fashion advice based on their facial and bone structure, making it easy to find the style that best suits them.

[1151] The processing flow will be explained below.

[1152] Step 1:

[1153] The user launches the smartphone app and takes a photo of their face and whole body, or selects an existing photo.

[1154] Step 2:

[1155] Users upload photos to the server through the app's interface, and the photo data is transmitted over secure communication.

[1156] Step 3:

[1157] The server receives photo data uploaded by the user.

[1158] Step 4:

[1159] The server passes the received photo data to the image processing module for analysis, where the face detection and feature extraction process begins.

[1160] Step 5:

[1161] The server's image processing module extracts facial shape and skeletal features, such as facial contours, eye, nose, and mouth positions, and skeletal proportions.

[1162] Step 6:

[1163] The server then compares the extracted feature data with a database of fashion models, which contains data on models with various facial and body shapes.

[1164] Step 7:

[1165] The server searches the database for a model that best matches the user's characteristics and calculates the optimal fashion style based on that model.

[1166] Step 8:

[1167] The server generates advice on optimal fashion items and styles, including specific clothing types and styling suggestions.

[1168] Step 9:

[1169] The server transmits the generated fashion advice to the user's terminal.

[1170] Step 10:

[1171] The user's device receives the advice data sent from the server and displays it within the application, allowing the user to check it and use it to help them choose their own fashion.

[1172] Example 1

[1173] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1174] Conventional fashion advice systems often have difficulty providing personalized advice that takes into account the user's facial and body shapes. Furthermore, they have faced the problem of requiring time and effort for users to find the fashion items and styling that best suit them. This makes it difficult for users to easily find the style that best suits them.

[1175] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1176] In this invention, the server includes means for allowing users to upload photos of their face and skeletal structure, means for receiving the uploaded photos and extracting facial and skeletal features, means for comparing the extracted features with a fashion model database to calculate the most suitable fashion for the user, means for generating advice on the most suitable fashion items and style and sending it to the user terminal, means for displaying the generated advice on a user interface, and means for performing a facial recognition algorithm, a machine learning algorithm, and database comparison. This allows users to receive accurate fashion advice based on their facial and skeletal features, making it easy to find the style that best suits them.

[1177] "User Device" means a device equipped with an application that allows a user to take and select photos and upload them to the server, and that has the functionality to display fashion advice sent from the server.

[1178] The "server" is a central processing unit that receives photos uploaded from the user's terminal, analyzes facial and skeletal features, compares the analysis results with a database of fashion models to calculate the most suitable fashion items and style for the user, and sends the generated advice to the user's terminal.

[1179] A "fashion model database" is a database that stores information on fashion models with various facial shapes and skeletal features, and is a data storage that includes each model's facial shape, body shape, recommended fashion style, item list, etc.

[1180] A "facial recognition algorithm" is an algorithm used to analyze the shape and features of a face from an uploaded photo, and is a technology that uses image processing libraries such as OpenCV and Dlib to detect facial contours and feature points.

[1181] A "machine learning algorithm" is an algorithm that performs classification and prediction based on extracted facial and skeletal feature data, and is a technology that analyzes feature data using methods such as Random Forest and SVM (Support Vector Machine).

[1182] "Fashion items" are specific clothing, accessories, and other fashion products that are suggested to a user and are recommended based on the user's facial and skeletal features.

[1183] A "prompt" is an instruction entered into a generative AI model to output appropriate fashion advice, and is text input to obtain optimal output based on the user's characteristics and requests.

[1184] MODE FOR CARRYING OUT THE INVENTION

[1185] Overall system configuration

[1186] This invention is a system that allows users to upload photos of their face and bone structure and provides fashion advice based on those photos. The system is composed of a user terminal, a server, and a database of fashion models.

[1187] User device functions

[1188] The user device has an application that allows users to take and select photos and upload them to the server. It also has a function to display fashion advice sent from the server. Users launch the app, take photos of their face and body using the camera, and then upload the photos to the server through the app interface.

[1189] Examples:

[1190] The user launches the app on their smartphone, takes a photo of their face and whole body, and then clicks the app's upload button to send the photo to the server.

[1191] Server Features

[1192] The server receives photos uploaded from the user's device and analyzes facial and skeletal features. This analysis uses image processing technology and machine learning algorithms. Specifically, OpenCV and Dlib are used to detect facial shape and feature points, and the data is analyzed using machine learning algorithms such as Random Forest and SVM. After analysis, the results are compared with a fashion model database to calculate the optimal fashion items and style for the user. Final advice is then sent to the user's device.

[1193] Examples:

[1194] The server receives the photo sent by the user and performs facial recognition using OpenCV. It extracts facial feature points using Dlib and then applies the Random Forest algorithm to identify the user's facial and skeletal characteristics. It then compares the results with a fashion model database to select a model that most closely matches the "long face, slim body shape" description and calculates the model's recommended fashion style.

[1195] Fashion Model Database Features

[1196] This database stores information on fashion models with various facial shapes and bone structures. The database includes each model's facial shape, body shape, recommended fashion style, and item list.

[1197] Examples:

[1198] If a model with a "round face and plump figure" is registered in the database, the style and items that model recommends (e.g., A-line dresses, flared pants, etc.) are recorded.

[1199] Fashion advice generation and transmission to user devices

[1200] After the server selects the optimal model, it generates specific fashion advice based on the model's data. The advice includes recommended fashion items and styling details. The advice is sent to the user's device and displayed on a user interface.

[1201] Examples:

[1202] Based on the user's characteristics, specific styling advice is generated, such as "V-neck tops and high-waisted pants look good on you." This advice is sent to the user's device and displayed within the app.

[1203] Prompt Sentence Examples

[1204] A typical example of a prompt to be input to a generative AI model might be:

[1205] "Based on photos of the user's face and body, please suggest the most suitable fashion items and styles. The user's characteristics are as follows: 'Long face, slim body'. Please output the most suitable fashion advice for this."

[1206] Using this prompt, the generative AI model outputs content that provides appropriate fashion advice.

[1207] The system allows users to receive accurate fashion advice based on their facial and bone structure, making it easy to find the style that best suits them.

[1208] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1209] Step 1:

[1210] Receiving and storing photos on the server

[1211] The server receives photos uploaded from the user's device. This input is a face and full-body photo sent by the user through the app. The server saves the received photos in temporary storage. As a concrete example, the server receives image data through an HTTP request and saves the data in local storage.

[1212] Input: Face and full-body photo data sent from the user device.

[1213] Data processing: Analyzing HTTP requests and saving image data.

[1214] Output: Saved image file.

[1215] Step 2:

[1216] Server-based face recognition and skeletal feature extraction

[1217] The server analyzes the stored photos to extract facial shape and skeletal features. This step uses image processing technologies such as OpenCV and Dlib. First, a facial recognition algorithm is applied, and then Dlib is used to extract the coordinates of facial feature points (eyes, nose, mouth, etc.).

[1218] Input: Saved face and full-body image files.

[1219] Data processing: Face recognition using OpenCV and feature point extraction using Dlib.

[1220] Output: Facial shape and skeletal feature coordinate data.

[1221] Step 3:

[1222] Application of machine learning algorithms by the server

[1223] The server inputs the extracted facial and skeletal feature data into a machine learning algorithm to classify the user's face shape and body shape. The algorithms used include Random Forest and SVM. In this step, the feature data is input into the machine learning model as a numerical vector to obtain the classification results.

[1224] Input: Coordinate data of facial shape and skeletal features.

[1225] Data processing: Vectorizing feature data and inputting it into a machine learning model.

[1226] Output: Classified face shape and body shape data.

[1227] Step 4:

[1228] Database verification by the server

[1229] The server compares the classified face shape and body shape data with a fashion model database, which contains each model's face shape, body shape, recommended fashion style, and items. In this step, the server searches for the most suitable model and obtains the model's data.

[1230] Input: Classified face shape and body shape data.

[1231] Data processing: Search the fashion model database and select the most suitable model.

[1232] Output: Data of the best fashion model.

[1233] Step 5:

[1234] Server-based fashion advice generation

[1235] The server generates specific fashion advice based on the data of the selected fashion model. Using the generative AI model, it generates text that suggests the most suitable fashion items and styles for the user. For example, it might generate advice such as "V-neck tops and high-waisted pants look good on you."

[1236] Input: Data of the optimal fashion model.

[1237] Data processing: Generate text advice using a generative AI model.

[1238] Output: Text data of fashion advice.

[1239] Step 6:

[1240] Server sends advice and displays it on the user's device

[1241] The server sends the generated fashion advice to the user's device, which then displays the received advice within the application. Specifically, the server sends the advice as an HTTP response, and the user's device analyzes it and displays it on the interface.

[1242] Input: Text data of fashion advice.

[1243] Data processing: Sending advice and analyzing received data.

[1244] Output: User interface with advice displayed.

[1245] (Application example 1)

[1246] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1247] Fashion-conscious users face challenges in easily finding the styles and items that best suit them. In particular, there are few methods for quickly providing personalized fashion advice based on a user's face shape and bone structure, making it time-consuming and labor-intensive to select a style that suits them. Furthermore, there is no way to immediately purchase products based on the advice provided, which can disrupt the user's shopping experience. To address these issues, a system is needed that can quickly and accurately analyze a user's characteristics, provide optimal fashion advice, and enable the user to purchase products based on the advice.

[1248] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1249] In this invention, the server includes means for allowing a user to upload images of their face and bone structure, means for receiving the uploaded images and extracting facial and bone structure features, means for comparing the extracted features with a fashion model database to calculate the most suitable fashion for the user, means for generating advice on the most suitable fashion items and style and transmitting it to the user terminal, means for displaying the generated advice on a user interface, and means for allowing the user to purchase products based on the advice through an application. This allows the user to receive the most suitable fashion advice based on their own features and instantly purchase products based on the advice.

[1250] A "user terminal" is a device that allows a user to take photos, upload images to a server through an application, and receive fashion advice.

[1251] The "server" is a computer system that receives images sent from a user terminal, analyzes them, generates fashion advice, and sends it to the user terminal.

[1252] "Image processing technology" is a technology for analyzing images taken by a user and extracting facial and skeletal features.

[1253] "Machine learning algorithms" are artificial intelligence technologies that compare facial and skeletal features with a database to generate optimal fashion advice for users.

[1254] A "fashion model database" is a collection of information that stores data on models based on various face shapes and body shapes, and is used to provide fashion advice based on that data.

[1255] The "user interface" refers to a screen or operating means for displaying the generated fashion advice to the user.

[1256] A "prompt sentence" is text data that uses a generative AI model to interactively analyze the user's image and features.

[1257] The "shopping function" is a function that allows a user to directly purchase items shown in the advice based on the generated fashion advice.

[1258] This invention is a system that allows users to upload images of their face and bone structure and provides fashion advice based on those images. The system consists of a user terminal, a server, and a database of fashion models.

[1259] User device functions

[1260] The user terminal has a function that allows the user to take pictures and upload them to the server through the application, and also provides an interface that displays fashion advice sent from the server and allows the user to purchase products based on that advice.

[1261] For example, a user launches the app and takes a picture of their face and whole body using their camera. They then upload the image to a server through the app's interface. After uploading, the user waits for a response from the server and can view the resulting advice on the screen. Based on this advice, a link or button to purchase the product is provided, allowing the user to continue shopping.

[1262] Server Features

[1263] The server receives the images sent from the user terminal and extracts facial and skeletal features using image processing techniques and machine learning algorithms.

[1264] Specifically, the server does the following:

[1265] 1. Image reception: Has an API endpoint for receiving image files sent from the user device.

[1266] 2. Feature extraction: The received image is analyzed and facial and skeletal features are extracted using a facial recognition algorithm, such as the Python face_recognition library. Numerical data such as facial shape and skeletal proportions are obtained.

[1267] 3. Database Matching: The extracted features are matched against a database of fashion models, which contains feature data for each model and fashion styles based on those features.

[1268] 4. Advice Generation: Select the best model and generate styling advice based on that model. This advice is crafted in detail using the generative AI model and prompt text.

[1269] 5. Sending advice: Install an API to send the generated advice to the user's device.

[1270] Fashion Model Database Features

[1271] The fashion model database stores information on models with various facial shapes and bone structures. The database includes each model's facial shape, body shape, recommended fashion style, and item list.

[1272] For example, if a model with a "round face and plump figure" is registered in the database, the style and clothing that model recommends (e.g., A-line dresses, flared pants, etc.) are recorded. This allows the server to provide advice that best suits the user's characteristics.

[1273] Fashion advice generation and transmission to user devices

[1274] After the server selects the optimal model, it generates specific fashion advice based on the model's data. The advice includes recommended fashion items and styling details. The advice is sent to the user's device and displayed on a user interface.

[1275] For example, styling advice such as "V-neck tops and high-waisted pants look good on you" is generated based on the user's characteristics. This advice is sent to the user's device and displayed within the app. The user can also purchase items directly within the app.

[1276] Prompt Sentence Examples

[1277] "Analyze the user's photo and extract facial shape and bone structure features. Based on those features, provide the most suitable fashion items and styling advice."

[1278] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1279] Step 1:

[1280] The user launches the smartphone app and takes a photo of their face and whole body.

[1281] In this step, the user takes a photo using the app's camera function and the photo is saved in the smartphone. The input is the original photo taken by the user, and the output is the photo data saved in the smartphone.

[1282] Step 2:

[1283] The user terminal uploads the photograph to the server.

[1284] In this step, when a user clicks the upload button in the app, the photo file is sent to the server through the HTTP request process within the app. The input is the photo data stored on the smartphone, and the output is the data sent to the server.

[1285] Step 3:

[1286] The server receives the photo data and uses image processing techniques and machine learning algorithms to extract facial and skeletal features.

[1287] Specifically, the server analyzes the photo using Python's face_recognition library and extracts the facial shape and skeletal features as numerical data. The input is the photo data sent to the server, and the output is the extracted facial and skeletal feature data.

[1288] Step 4:

[1289] Based on the extracted feature data, the server compares it with a database of fashion models and calculates the most suitable fashion for the user.

[1290] The server uses the extracted feature data to compare it with the features of each model stored in the database and selects the optimal model. The input is the extracted feature data and a database of fashion models, and the output is the optimal model data and styling advice.

[1291] Step 5:

[1292] The server uses the generated AI model and prompts to generate specific fashion advice.

[1293] In this step, the server inputs prompts into the generative AI model to generate specific fashion item and style advice based on the user's characteristics. The input is the optimal model's data and prompts, and the output is specific fashion advice.

[1294] Step 6:

[1295] The server transmits the generated advice to the user terminal.

[1296] The server sends the generated advice to the user terminal as an HTTP response. The input is the specific fashion advice, and the output is the advice data sent to the user terminal.

[1297] Step 7:

[1298] The user terminal displays the received advice on a user interface, and the user purchases the product based on the advice.

[1299] In this step, the user can review the advice displayed within the app and purchase the product by clicking a purchase link or button. The input is the advice data received from the server, and the output is the advice and purchase procedure displayed on the user's display screen.

[1300] Prompt Sentence Examples

[1301] "Analyze the user's photo and extract facial shape and bone structure features. Based on those features, provide the most suitable fashion items and styling advice."

[1302] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1303] Overall system configuration

[1304] This invention is a system that allows users to upload photos of their face and bone structure, and provides fashion advice based on the user's emotions. The system consists of a user terminal, a server, an emotion engine, and a database of fashion models.

[1305] User device functions

[1306] The user device is equipped with an application that allows users to take and select photos and upload them to the server, and also has the function of displaying fashion advice and personalized recommendation advice based on emotions sent from the server.

[1307] Example: A user launches an app, uses the camera to take a photo of their face and body, captures facial expressions to recognize their current emotion, and then uploads the photo to a server through the app interface.

[1308] Server Features

[1309] The server receives photos and facial expression data uploaded from the user's device and analyzes facial and skeletal features and emotions. The analysis uses image processing technology, machine learning algorithms, and an emotion engine. After analysis, the server compares the results with a database of fashion models and calculates advice based on the user's optimal fashion items, style, and emotions. The final advice is then sent to the user's device.

[1310] Example: The server receives the uploaded photo and facial expression data, and applies facial recognition and emotion recognition algorithms to extract facial shape, skeletal features, and the current emotional state (e.g., "joy" or "sadness"). The extracted data is compared with a database to select the most suitable fashion model.

[1311] Emotion Engine Functions

[1312] The emotion engine is a module for recognizing emotions from facial expression data uploaded by users. This engine uses machine learning algorithms to analyze the user's facial expressions and identify the emotion at that time.

[1313] Example: If the emotion engine recognizes the emotion "joy" from a user's facial expression, it will recommend fashion items that match that emotion, including relaxed styles and cheerful color palettes.

[1314] Fashion Model Database Features

[1315] This database stores information on fashion models with various facial shapes and bone structures. The database includes each model's facial shape, body shape, recommended fashion style, and item list.

[1316] Example: If a model with a "round face and plump figure" is registered in the database, the style and items recommended by that model (e.g., A-line dresses, flared pants, etc.) will be recorded.

[1317] Fashion advice generation and transmission to user devices

[1318] After the server selects the optimal model, it generates specific fashion advice based on the model's data and the user's emotions. The advice includes recommended fashion items, styling details, and emotional suggestions. The generated advice is sent to the user's device and displayed on a user interface.

[1319] Example: Based on the user's characteristics and emotions, specific styling advice such as "V-neck tops and high-waisted pants look good on you" is provided, along with additional emotional advice such as "Bright colors are recommended for today's mood." This advice is sent to the user's device and displayed within the app.

[1320] Use Case Details

[1321] For example, if a user has a "long face, slim figure," and is currently feeling "happy," the server will extract the corresponding features and emotions from the database and determine that "Model A" is the best fit. Model A recommends that "V-neck tops and high-waisted pants look good on you," so in addition to this styling advice, it will provide the user with emotion-based advice such as "bright-colored items are perfect for your mood today." Users can check these advice within the app and use them as reference for shopping and outfits.

[1322] The system allows users to receive accurate fashion advice based on their facial and skeletal features and emotions, making it easy to find the style that best suits them.

[1323] The processing flow will be explained below.

[1324] Step 1:

[1325] The user launches the smartphone app and takes a photo of their face, their whole body, and their facial expression, or selects an existing photo.

[1326] Step 2:

[1327] Users upload photo data and facial expression data to the server through the app interface, and the data is transmitted over secure communication.

[1328] Step 3:

[1329] The server receives the photo data and facial expression data uploaded by the user.

[1330] Step 4:

[1331] The server then passes the received photo data to the image processing module for analysis, where the process of recognizing facial and skeletal features and facial expressions begins.

[1332] Step 5:

[1333] The server's image processing module extracts facial shape and skeletal features, such as facial contours, eye, nose, and mouth positions, and skeletal proportions.

[1334] Step 6:

[1335] Furthermore, an emotion engine analyzes the facial expression data to identify the user's current emotion, such as "happiness," "sadness," or "surprise."

[1336] Step 7:

[1337] The server compares the extracted feature data and emotion data with a database of fashion models, which contains data on models with various facial and body shapes.

[1338] Step 8:

[1339] The server searches the database for a model that best matches the user's facial and skeletal features and emotional data, and calculates the optimal fashion style based on that model.

[1340] Step 9:

[1341] The server generates advice about the calculated fashion items and styles, including recommended clothing types, styling details, and additional suggestions depending on the emotion.

[1342] Step 10:

[1343] The server sends the generated fashion advice to the user's device, including colors and styling that match the user's emotions.

[1344] Step 11:

[1345] The user's device receives the advice data sent from the server and displays it within the application, allowing the user to check it and use it to help them choose their own fashion.

[1346] Step 12:

[1347] Based on the advice provided, the user purchases the recommended fashion items or puts together an outfit.

[1348] Example 2

[1349] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1350] Conventional fashion advice systems only consider the user's facial and skeletal features, making it difficult to provide advice that reflects the user's current emotions. Furthermore, advice that does not consider emotions makes it difficult to select the optimal fashion that matches the user's feelings. Therefore, there is a need for a method to support users in selecting fashion that suits their daily mood and the occasion.

[1351] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1352] In this invention, the server includes a means for allowing a user to upload a photograph of their face and skeletal structure, a means for receiving the uploaded photograph and extracting facial and skeletal features, and a means for analyzing the user's emotions using an emotion engine, thereby enabling optimal fashion advice to be provided to the user based on the extracted features and analyzed emotions.

[1353] "User" refers to any person who uses the system to upload their own photos and receive fashion advice.

[1354] "Server" refers to a computer system that receives data uploaded by users and performs analysis and advice generation.

[1355] "Photo" refers to image data including a user's face and bone structure that is uploaded by the user.

[1356] "Facial and skeletal features" refers to data being analyzed that indicates the user's physical characteristics, such as facial shape and skeletal structure.

[1357] "Emotion engine" refers to a module that analyzes and identifies emotions from a user's facial expression data.

[1358] A "fashion model database" refers to an information repository that stores data on each model based on their face shape and body type.

[1359] "Means for generating advice" refers to the process of creating recommendations for the most suitable fashion items and styles for the user based on the analyzed features and emotional data.

[1360] "User Terminal" refers to the device used by a User to upload photos and receive and view advice.

[1361] "User interface" refers to the screen and interaction mechanism that displays advice generated on the user's terminal and allows the user to operate it.

[1362] "Image processing techniques" refers to techniques used to extract facial and skeletal features from photographs.

[1363] "Machine learning algorithms" refer to computational techniques used for data analysis and emotion recognition.

[1364] The system of this invention allows users to upload photos of their face and bone structure, and provides appropriate fashion advice based on the user's emotions. The system is composed of a user terminal, a server, an emotion engine, and a database of fashion models.

[1365] User device functions

[1366] The user device is equipped with an application that allows users to take and select photos and upload them to the server, and also has the function of displaying fashion advice and personalized recommendation advice based on emotions sent from the server.

[1367] Specifically, the user launches the app and takes a photo of their face and whole body using the camera. Furthermore, facial expressions are captured to recognize the current emotion, and the captured images are uploaded to a server through the app's interface. The hardware used is a mobile device such as a smartphone or tablet, and the app is developed based on a general-purpose application such as a "movie viewer for mobile devices."

[1368] Server Features

[1369] The server receives photos and facial expression data uploaded from the user's device and analyzes facial and skeletal features and emotions. Image processing technology, image processing libraries such as "OpenCV," and machine learning algorithms such as "DeepFace" are used for the analysis. The server then compares the analysis results with a database of fashion models to calculate advice based on the user's optimal fashion items, style, and emotions. The final advice is then sent to the user's device.

[1370] Specifically, the server receives a photo and facial expression data from the user, extracts facial shape and skeletal features, and simultaneously recognizes the user's emotional state (e.g., "joy" or "sadness"). The extracted data is compared with a database of fashion models, and suggestions for optimal fashion items, styling details, and emotions are generated. This data is then sent to the user's device, where the user can view it within the app.

[1371] Emotion Engine Functions

[1372] The emotion engine is a module for recognizing emotions from facial expression data uploaded by users. This engine uses machine learning algorithms to analyze the user's facial expressions and recognize specific emotions.

[1373] For example, if the emotion engine recognizes the emotion of "joy" from the user's facial expression, it will recommend fashion items that match that emotion, including a relaxed style and cheerful color palette. The machine learning algorithms used can be "TensorFlow" or "Keras."

[1374] Fashion Model Database Features

[1375] The fashion model database stores information on fashion models with various facial shapes and skeletal features. The database includes each model's facial shape, body shape, recommended fashion style, and item list.

[1376] Specifically, if a model with a "round face and plump figure" is registered in the database, the style and items recommended by that model (e.g., A-line dress, flared pants, etc.) are recorded.

[1377] Fashion advice generation and transmission to user devices

[1378] After the server selects the optimal model, it generates specific fashion advice based on the model's data and the user's emotions. The advice includes recommended fashion items, styling details, and emotional suggestions. The generated advice is sent to the user's device and displayed on a user interface.

[1379] As a specific example, based on the user's characteristics and emotions, in addition to specific styling advice such as "V-neck tops and high-waisted pants look good on you," additional emotional advice such as "bright colors are recommended for today's mood" is provided.

[1380] Prompt Sentence Examples

[1381] "You will create a program that allows users to upload photos of their face and bone structure, recognizes emotions, and provides optimal fashion advice. Using image processing technology and machine learning algorithms, you will design a system that analyzes facial and bone structure features and emotions, and compares them with a database of fashion models to generate advice."

[1382] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1383] Step 1:

[1384] Users take or select a photo of their face and bone structure and upload it to a server through the app.

[1385] Specifically, the user launches the app and taps the "Take a Photo" button. The user then uses the camera to take a photo of their face and whole body, and taps the "Upload" button afterward. The input is the user's photo data, and the output is the photo data to be sent to the server.

[1386] Step 2:

[1387] The server receives the photo data uploaded by the user and temporarily stores it.

[1388] Specifically, the server receives photo data via an HTTP request and temporarily stores it in a database. The input is photo data from the user's device, and the output is the image data stored on the server.

[1389] Step 3:

[1390] The server analyzes the stored photographic data and extracts facial and skeletal features.

[1391] Specifically, the server uses OpenCV to perform facial recognition and extract skeletal features. It applies image processing algorithms to detect facial contours and skeletal points. The input is temporarily saved image data, and the output is facial and skeletal feature data.

[1392] Step 4:

[1393] The emotion engine analyzes facial expression data and recognizes specific emotions.

[1394] Specifically, facial expression data is fed into a machine learning algorithm such as "DeepFace" to recognize emotions (e.g., "happiness" or "sadness"). The input is facial expression data, and the output is emotional data.

[1395] Step 5:

[1396] The server compares the extracted facial and skeletal feature data and emotion data with a database of fashion models.

[1397] Specifically, the server issues a query to the fashion model database to retrieve data on models with similar features and emotions. The input is feature data and emotion data, and the output is the optimal model data.

[1398] Step 6:

[1399] The server generates specific fashion advice based on the acquired model data and the user's emotional data.

[1400] Specifically, the server analyzes the recommended fashion items and styling details based on the acquired model data, and generates additional suggestions based on the user's emotions. The inputs are model data and emotion data, and the output is generated advice data.

[1401] Step 7:

[1402] The generated advice data is sent to the user terminal, and the user confirms the advice through the app.

[1403] Specifically, the server sends the generated advice data to the user device via the endpoint. The user device receives the advice data from the endpoint and displays it on the app interface. The input is the generated advice data, and the output is the advice displayed on the user device.

[1404] (Application example 2)

[1405] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1406] Conventional fashion advice systems suggest styles based on the user's facial and skeletal features, but are unable to consider the user's emotional state. This makes it difficult to provide more personalized advice tailored to the user's mood. It is also difficult to provide real-time fashion advice in physical stores. Therefore, there is a need for the development of a system that can consider the user's emotions and provide optimal fashion advice in real time.

[1407] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1408] In this invention, the server includes means for allowing a user to upload a photograph of their face and skeletal structure, means for receiving the uploaded photograph and extracting facial and skeletal features and emotions, means for comparing the extracted features and emotions with a fashion model database to calculate the most suitable fashion for the user, means for generating advice on the most suitable fashion items and style and sending it to the user terminal together with additional suggestions based on the emotions, and means for displaying the generated advice on a user interface and a display device, thereby enabling the provision of personalized fashion advice that takes the user's emotions into consideration.

[1409] "Means for enabling users to upload photographs of their own face and bones" refers to a function for taking or selecting photographs of the face and bones using the user terminal and sending them to the server.

[1410] "Means for the server to receive uploaded photos and extract facial and skeletal features and emotions" refers to technology in which the server analyzes image data sent from the user terminal and identifies the facial shape, skeletal features, and current emotional state.

[1411] "Means for comparing the extracted features and emotions with a database of fashion models and calculating the most suitable fashion for the user" refers to a function that recommends the most suitable fashion style and items from a database of fashion models based on facial and skeletal features and emotions.

[1412] "Means for generating advice regarding optimal fashion items and styles and transmitting the advice to a user terminal together with additional suggestions based on emotions" refers to a technology for generating additional suggestions tailored to the user's emotional state along with fashion advice and transmitting the same from a server to a user terminal.

[1413] "Means for displaying generated advice on a user interface and display device" refers to functionality for visually presenting generated fashion advice and emotion-based suggestions on a user terminal and other display devices.

[1414] "Means using image processing technology and machine learning algorithms" refers to technology that uses image processing software and machine learning models to analyze image data and perform accurate feature extraction and emotion recognition.

[1415] A "fashion model database" refers to a data storage that accumulates information on various face shapes, body types, fashion items, and styles.

[1416] This system allows users to upload photos of their face and bone structure in a physical store, and provides fashion advice based on the user's emotions. The system consists of a user terminal, a server, an emotion engine, and a fashion model database.

[1417] Overall system configuration

[1418] User device functions

[1419] The user device is equipped with an application that allows users to take and select photos and upload them to a server. It also has the function of displaying fashion advice sent from the server and personalized recommendations based on emotions. Specifically, this corresponds to smart glasses or head-mounted displays installed in fitting rooms, etc.

[1420] Server Features

[1421] The server receives photos and facial expression data uploaded from the user's device and analyzes facial and skeletal features and emotions. The analysis uses image processing techniques (e.g., OpenCV), machine learning algorithms (e.g., TensorFlow / Keras), and an emotion engine (EmotionEngine). After analysis, the server compares the results with a database of fashion models and calculates advice based on the user's optimal fashion items, style, and emotions. The final advice is sent to the user's device.

[1422] Emotion Engine Functions

[1423] The Emotion Engine is a module that recognizes emotions from facial expression data uploaded by users. This engine uses machine learning algorithms to analyze the user's facial expressions and identify the emotion at that time. For example, if the Emotion Engine recognizes the emotion "joy" from the user's facial expression, it will recommend fashion items that match that emotion, including a relaxed style and cheerful color palette.

[1424] Fashion Model Database Features

[1425] This database stores information on fashion models with various facial shapes and skeletal features. The database includes information on each model's facial shape, body type, recommended fashion style, and item list. Specifically, if a model with a "round face and plump body" is registered, the style and items recommended by that model (e.g., A-line dresses, flared pants, etc.) will be recorded.

[1426] Fashion advice generation and transmission to user devices

[1427] After the server selects the optimal model, it generates specific fashion advice based on the model's data and the user's emotions. The advice includes recommended fashion items, styling details, and emotional suggestions. The generated advice is sent to the user's device and displayed on a user interface.

[1428] Explanation of program processing

[1429] 1. Hardware and Software Use

[1430] Camera (e.g. Logitech C920): Takes a photo of the user.

[1431] Emotion Engine: Analyze emotions using machine learning (e.g. TensorFlow / Keras).

[1432] Fashion Advice Server: A server that analyzes facial features and bone structure (e.g., built with Flask).

[1433] Fashion Model Database: Provides fashion model information based on facial and skeletal features.

[1434] Display device (e.g., smart glasses, HMD): displays advice to the user (e.g., Google Glass, HoloLens).

[1435] 2. Data processing or data calculation

[1436] Image Capture: Take a picture of the user with the camera and store it locally.

[1437] Image analysis: Sends the image to an analysis server to analyze the facial shape, bone structure, and emotion.

[1438] Database matching: The analysis results are compared with a fashion model database to select the most suitable model and style.

[1439] Advice generation: Generate personalized fashion advice based on the analysis results and information obtained from the model.

[1440] Adding specific examples

[1441] For example, if a user's facial expression captured by a "smart mirror" is "surprised," their face shape is "round," and their body type is "slim," the device will display the following message in real time:

[1442] "If you have a round face, a V-neck top is a great choice. A surprised look is great, so add some lighter colors to make it stand out even more."

[1443] Example prompts to input to the generative AI model

[1444] Generate optimal fashion advice for a user with a round face, slim figure, and surprised expression.

[1445] Create a system that provides real-time fashion advice based on facial shape and emotion.

[1446] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1447] Step 1:

[1448] The user takes a photo.

[1449] Input: The user takes a photo of their face and skeletal structure using smart glasses or a head-mounted display.

[1450] Data processing: The camera captures a photo of the user's face and overall appearance, and stores it locally as an image file.

[1451] Output: The saved image file (e.g. user_image.jpg).

[1452] Step 2:

[1453] The user terminal uploads the photo to the server.

[1454] Input: A saved image file.

[1455] Data processing: The user device sends the image file to the server.

[1456] Output: Image data sent to the server.

[1457] Step 3:

[1458] The server receives the image data and extracts facial and skeletal features and emotions.

[1459] Input: Image data sent to the server.

[1460] Data processing:

[1461] Step 1: The server uses image processing technology (e.g., OpenCV) to analyze the facial shape and skeletal features.

[1462] Step 2: The server uses a machine learning algorithm (e.g., TensorFlow / Keras) to identify the emotional state (e.g., "surprise," "joy," etc.) from the image.

[1463] Output: Extracted face shape, skeletal features, and emotion data.

[1464] Step 4:

[1465] The server compares the extracted features and emotions with a fashion model database and calculates the most suitable fashion.

[1466] Input: Extracted face shape, skeletal features, and emotion data.

[1467] Data processing:

[1468] Operation 1: The server accesses a fashion model database (e.g., FashionModelDatabase) and searches for the corresponding model data.

[1469] Action 2: Based on the collated data, the best fashion items and styles are selected for the user.

[1470] Output: Data about the best fashion items and styles.

[1471] Step 5:

[1472] The server generates advice and sends it to the user terminal.

[1473] Input: Data about optimal fashion items and styles, and sentiment data.

[1474] Data processing:

[1475] Action 1: The server generates fashion advice, which includes selected fashion items and styles, as well as additional suggestions based on the user's emotions.

[1476] Operation 2: The generated advice is sent to the user terminal.

[1477] Output: Fashion advice sent to the user's device.

[1478] Step 6:

[1479] The user terminal displays the generated advice.

[1480] Input: Fashion advice sent to user device.

[1481] Data processing: Displaying the generated advice on a user interface and display device (e.g., smart glasses, HMD).

[1482] Output: Fashion advice that the user can visually see.

[1483] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the control target 443 to output the result of the specific processing. The microphone 238 acquires voice indicating a user input regarding the result of the specific processing. The control unit 46A transmits voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[1484] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1485] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the robot 414.

[1486] The emotion identification model 59 as an emotion engine may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to an emotion map (see FIG. 9), which is a specific mapping. Similarly, the emotion identification model 59 may determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[1487] FIG. 9 illustrates an emotion map 400 on which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. Emotions closer to the center of the concentric circles are more primitive. Emotions representing states and behaviors arising from a state of mind are arranged on the outer edges of the concentric circles. The concept of emotion includes both affect and mental states. Emotions generally generated from reactions occurring in the brain are arranged on the left side of the concentric circles. Emotions generally induced by situational judgment are arranged on the right side of the concentric circles. Emotions generally generated from reactions occurring in the brain and induced by situational judgment are arranged on the upper and lower sides of the concentric circles. Furthermore, the emotion of "pleasure" is arranged on the upper side of the concentric circles, and the emotion of "discomfort" is arranged on the lower side. In this way, in the emotion map 400, multiple emotions are mapped based on the structure by which emotions are generated, and emotions that tend to occur simultaneously are mapped close to each other.

[1488] These emotions are distributed in the 3 o'clock direction on emotion map 400, and typically fluctuate between relief and anxiety. In the right half of emotion map 400, situational awareness dominates over internal sensations, resulting in a sense of calm.

[1489] The inside of emotion map 400 represents what is going on in the mind, and the outside of emotion map 400 represents behavior, so the further you go outside emotion map 400, the more visible the emotions become (the more they are expressed in behavior).

[1490] Human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. Emotions can also be created for robots, automobiles, and motorcycles, based on various balances, such as posture and remaining battery life. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. An emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on Voice Emotion Recognition and Emotional Brain Physiological Signal Analysis Systems, Tokushima University, Doctoral Dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map lists emotions belonging to the "reaction" domain, where sensation is dominant. The right half of the emotion map lists emotions belonging to the "situation" domain, where situational awareness is dominant.

[1491] The emotion map defines two emotions that promote learning. One is a negative emotion on the situation side, around the middle of "repentance" or "reflection." In other words, this occurs when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is a positive emotion on the response side, around "desire." In other words, this occurs when the robot experiences positive feelings such as "I want more" or "I want to know more."

[1492] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​indicating each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple pieces of training data that are combinations of user input and emotion values ​​indicating each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions that are located close to each other have similar values, as in the emotion map 900 shown in FIG. 10. FIG. 10 shows an example in which multiple emotions, "relieved," "calm," and "reassuring," have similar emotion values.

[1493] The system according to the present disclosure has been described above mainly with respect to the functions of the data processing device 12, but the system according to the present disclosure is not necessarily implemented on a server. The system according to the present disclosure may be implemented as a general information processing system. The present disclosure may be implemented, for example, as a software program running on a personal computer or an application running on a smartphone, etc. The method according to the present disclosure may be provided to users in the form of SaaS (Software as a Service).

[1494] In the above embodiment, an example was given in which the specific processing is performed by one computer 22, but the technology of the present disclosure is not limited to this, and the specific processing may be distributed and performed by a plurality of computers including the computer 22. For example, the data generation model 58 may be provided in an external device of the data processing device 12, and data may be generated in the external device in accordance with input data.

[1495] In the above embodiment, an example in which the specific processing program 56 is stored in the storage 32 has been described, but the technology of the present disclosure is not limited to this. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-transitory storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-transitory storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes the specific processing in accordance with the specific processing program 56.

[1496] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[1497] It is not necessary to store all of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store all of the specific processing program 56 in the storage 32; only a portion of the specific processing program 56 may be stored.

[1498] The hardware resource for executing a specific process can be any of the following processors: An example of a processor is a CPU, which is a general-purpose processor that functions as a hardware resource for executing a specific process by executing software, i.e., a program. Another example of a processor is a dedicated electrical circuit, such as an FPGA (Field-Programmable Gate Array), a PLD (Programmable Logic Device), or an ASIC (Application Specific Integrated Circuit), which is a processor with a circuit configuration designed specifically for executing a specific process. Each processor has built-in or connected memory, and each processor uses the memory to execute the specific process.

[1499] The hardware resource that executes the specific processing may be configured with one of these various processors, or may be configured with a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Also, the hardware resource that executes the specific processing may be a single processor.

[1500] As an example of a system configured with a single processor, first, one processor is configured by combining one or more CPUs and software, and this processor functions as a hardware resource that executes a specific process. Second, there is a system that uses a processor that realizes the functions of an entire system including multiple hardware resources that execute a specific process on a single IC chip, as typified by SoC (System-on-a-chip). In this way, a specific process is realized using one or more of the above-mentioned various processors as hardware resources.

[1501] Furthermore, the hardware structure of these various processors can be, more specifically, an electric circuit that combines circuit elements such as semiconductor devices. The specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps may be deleted, new steps may be added, or the processing order may be rearranged, without departing from the spirit of the invention.

[1502] The above-described description and illustrations are a detailed explanation of the parts related to the technology of the present disclosure and are merely an example of the technology of the present disclosure. For example, the above description of the configuration, functions, actions, and effects is an explanation of an example of the configuration, functions, actions, and effects of the parts related to the technology of the present disclosure. Therefore, it goes without saying that unnecessary parts may be deleted, new elements may be added, or replacements may be made to the above-described description and illustrations within the scope of the gist of the technology of the present disclosure. Furthermore, to avoid confusion and facilitate understanding of the parts related to the technology of the present disclosure, the above-described description and illustrations omit explanations of common technical knowledge that do not require particular explanation to enable the implementation of the technology of the present disclosure.

[1503] All publications, patent applications, and technical standards mentioned in this specification are herein incorporated by reference to the same extent as if each individual publication, patent application, or technical standard was specifically and individually indicated to be incorporated by reference.

[1504] The following is further disclosed regarding the above embodiment.

[1505] (Claim 1)

[1506] means for allowing a user to upload a photograph of their face and bone structure;

[1507] a server receiving the uploaded photograph and extracting facial and skeletal features;

[1508] A means for comparing the extracted features with a database of fashion models to calculate the most suitable fashion for the user;

[1509] means for generating and transmitting advice on optimal fashion items and styles to a user terminal;

[1510] means for displaying the generated advice on a user interface;

[1511] A system including:

[1512] (Claim 2)

[1513] 10. The system of claim 1, wherein the means for extracting facial and skeletal features uses image processing techniques and machine learning algorithms.

[1514] (Claim 3)

[1515] 2. The system according to claim 1, wherein the database of fashion models stores data for each model based on face shape and body shape.

[1516] "Example 1"

[1517] (Claim 1)

[1518] means for allowing a user to upload a photograph of their face and bone structure;

[1519] a server receiving the uploaded photograph and extracting facial and skeletal features;

[1520] A means for comparing the extracted features with a database of fashion models to calculate the most suitable fashion for the user;

[1521] means for generating and transmitting advice on optimal fashion items and styles to a user terminal;

[1522] means for displaying the generated advice on a user interface;

[1523] a facial recognition algorithm, a machine learning algorithm, and a means for performing database matching;

[1524] A system including:

[1525] (Claim 2)

[1526] 10. The system of claim 1, wherein the means for extracting facial and skeletal features uses image processing techniques and machine learning algorithms.

[1527] (Claim 3)

[1528] 2. The system according to claim 1, wherein the database of fashion models stores data for each model based on face shape and body shape.

[1529] "Application Example 1"

[1530] (Claim 1)

[1531] means for allowing a user to upload an image of their face and bone structure;

[1532] a server receiving the uploaded image and extracting facial and skeletal features;

[1533] A means for comparing the extracted features with a database of fashion models to calculate the most suitable fashion for the user;

[1534] means for generating and transmitting advice on optimal fashion items and styles to a user terminal;

[1535] means for displaying the generated advice in a user interface;

[1536] means for enabling a user to make a purchase based on the advice through the application;

[1537] A system including:

[1538] (Claim 2)

[1539] 10. The system of claim 1, wherein the means for extracting facial and skeletal features uses image processing techniques and machine learning algorithms.

[1540] (Claim 3)

[1541] The system described in claim 1 is characterized in that the fashion model database stores data for each model based on face shape and body shape, and enables matching of images entered by the user with prompt sentences using the generative AI model.

[1542] "Example 2: Combining Emotion Engines"

[1543] (Claim 1)

[1544] means for allowing a user to upload a photograph of their face and bone structure;

[1545] a server receiving the uploaded photograph and extracting facial and skeletal features;

[1546] means for analyzing a user's emotions using an emotion engine;

[1547] A means for comparing the extracted features and the analyzed emotions with a database of fashion models to calculate the most suitable fashion for the user;

[1548] means for generating and transmitting advice on optimal fashion items and styles to a user terminal;

[1549] means for displaying the generated advice on a user interface;

[1550] A system including:

[1551] (Claim 2)

[1552] 10. The system of claim 1, wherein the means for extracting facial and skeletal features uses image processing techniques and machine learning algorithms.

[1553] (Claim 3)

[1554] 2. The system according to claim 1, wherein the database of fashion models stores data for each model based on face shape and body shape.

[1555] "Application example 2 when combining emotion engines"

[1556] (Claim 1)

[1557] means for allowing a user to upload a photograph of their face and bone structure;

[1558] a server receiving the uploaded photo and extracting facial and skeletal features and emotions;

[1559] A means for comparing the extracted features and emotions with a database of fashion models to calculate the most suitable fashion for the user;

[1560] means for generating and transmitting advice on optimal fashion items and styles, including additional sentiment-based suggestions, to a user terminal;

[1561] means for displaying the generated advice on a user interface and display device;

[1562] A system including:

[1563] (Claim 2)

[1564] 2. The system of claim 1, wherein the means for extracting facial and skeletal features and emotions uses image processing techniques and machine learning algorithms.

[1565] (Claim 3)

[1566] 2. The system according to claim 1, wherein the database of fashion models stores data for each model based on face shape and body shape. [Explanation of symbols]

[1567] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Device 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robot< / url:> < / url:> < / url:> < / url:>

Claims

1. means for allowing a user to upload a photograph of their face and bone structure; a server receiving the uploaded photograph and extracting facial and skeletal features; A means for comparing the extracted features with a database of fashion models to calculate the most suitable fashion for the user; means for generating and transmitting advice on optimal fashion items and styles to a user terminal; means for displaying the generated advice on a user interface; A system including:

2. 10. The system of claim 1, wherein the means for extracting facial and skeletal features uses image processing techniques and machine learning algorithms.

3. 2. The system according to claim 1, wherein the database of fashion models stores data for each model based on face shape and body type.

Images