System

Abstract

A system is provided.SOLUTION: A system comprising: means for receiving image data of a customer; means for receiving hairstyle requirements of the customer; means for analyzing the image data to extract hair characteristics of the customer; means for generating an optimal hairstyle based on the customer requirements and the hair characteristics; and means for suggesting the generated hairstyle to the customer.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] In traditional hair salons, customers often lacked the information they needed to choose the hairstyle that best suited them, and suggestions were limited by the skill and experience of the hairdresser. Furthermore, when customers tried out a new hairstyle or coloring, it was difficult for them to accurately grasp the results until they actually tried it, resulting in many cases of dissatisfaction. These issues led to a decline in customer satisfaction, making it necessary for hair salons to improve their service quality. [Means for solving the problem]

[0005] In order to solve the above problems, the present invention provides the following means. First, a means for receiving image data of a customer is provided. Next, a means for receiving a customer's hairstyle request is provided. Furthermore, a means for analyzing the received image data and extracting the customer's hair characteristics is provided. Next, a means for generating an optimal hairstyle based on the extracted hair characteristics and the customer's request is provided. Finally, a means for proposing the generated hairstyle to the customer is provided. This allows the customer to easily find a hairstyle that suits them. Furthermore, by including a means for performing a virtual coloring simulation for the generated hairstyle and a means for collecting the latest trend information and proposing additional hairstyles based on the trends, it is possible to make a variety of suggestions that meet customer expectations, thereby improving customer satisfaction.

[0006] "Means for receiving customer image data" refers to a process or device that electronically receives customer-provided photographs or images.

[0007] "Means for receiving customer hairstyle requests" refers to a process or device that collects and receives as input detailed information about a customer's desired hairstyle or styling.

[0008] "Means for analyzing image data to extract characteristics of the customer's hair" refers to technical means or algorithms for analyzing received image data to identify characteristics such as hair length, volume, texture, and color.

[0009] "Means for generating an optimal hairstyle based on a customer's requests and hair characteristics" refers to a system or algorithm that automatically designs or suggests an optimal hairstyle by combining styling elements desired by the customer with hair characteristics obtained through image analysis.

[0010] "Means for suggesting generated hairstyles to customers" refers to a process or device for presenting hairstyle options generated by the system to customers and conveying that information visually or otherwise.

[0011] "Means for performing virtual coloring simulation" refers to a system or algorithm that uses image processing technology to simulate various colorings on an image of a customer's hair and displays or provides the results.

[0012] "Means for collecting the latest trend information and suggesting additional hairstyles based on the trends to customers" refers to a process or system that collects information on the latest hairstyles and fashion trends from external data sources and internal databases and, based on this information, suggests the latest hairstyles that are suitable for customers. [Brief explanation of the drawings]

[0013] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11]FIG. 3 is a sequence diagram showing a processing flow of the data processing system according to the first embodiment. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0014] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0015] First, the terms used in the following description will be explained.

[0016] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0017] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0018] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0019] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0020] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0021] [First embodiment]

[0022] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0023] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0024] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0025] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0026] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0027] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0028] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0029] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0030] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0031] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0032] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0033] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0034] This invention is a system for building an application for beauty salons that proposes customized hairstyles that meet customer needs. This system analyzes the customer's image data, generates the optimal hairstyle, and provides coloring simulations and the latest trend information.

[0035] Overall system configuration

[0036] The system consists of the following main components:

[0037] 1. Photo upload system (terminal): A user interface for customers to upload their own images.

[0038] 2. Request input system (terminal): An interface for customers to input desired styling elements and specific requests.

[0039] 3. Data analysis system (server): Technology for analyzing received image data and extracting the characteristics of the customer's hair.

[0040] 4. Generation system (server): An algorithm for generating the optimal hairstyle based on the extracted hair characteristics and customer requests.

[0041] 5. Proposal system (terminal): An interface for proposing the generated hairstyle to the customer.

[0042] 6. Coloring Simulation System (Server): Technology that simulates virtual coloring.

[0043] 7. Trend information collection and suggestion system (server): A function that collects the latest trend information and suggests additional hairstyles based on it.

[0044] Program Processing Overview

[0045] 1. Upload photos and input requests (device processing)

[0046] The customer launches the application and logs in. After logging in, a photo upload screen appears, where the customer can take or select a photo of themselves. Next, a screen appears where the customer can enter information such as the amount, texture, length, and style of their desired hair. This data is then sent from the device to the server.

[0047] 2. Data analysis (server processing)

[0048] The server stores the photo data and requests received from the device in a database, and then uses image recognition technology to extract the customer's hair characteristics, such as hair length, volume, texture, and color.

[0049] 3. Hairstyle generation (server processing)

[0050] The server uses an AI model to generate the optimal hairstyle based on the analyzed hair characteristics and the customer's requests, and the generated style is stored in an internal database.

[0051] 4. Collecting trend information and making additional suggestions (server processing)

[0052] The server uses external APIs and internal databases to collect the latest hairstyle trends, and then provides additional hairstyle suggestions based on these trends.

[0053] 5. Coloring simulation (server processing)

[0054] The server then performs a virtual coloring simulation on the generated hairstyle, using an AI model to generate and save color variations that match the hair color and skin tone.

[0055] 6. Sending the proposal results (server processing)

[0056] The server sends the hairstyle proposal results and coloring simulation results to the terminal, which receives them and displays them to the customer.

[0057] 7. Displaying and Feedback of Proposal Results (Device Processing)

[0058] The customer checks the proposed hairstyles and color variations on the terminal and inputs their selection or feedback, which is then sent to the server.

[0059] Specific examples

[0060] Example 1: Proposing a new hairstyle to Customer A

[0061] 1. User uploads a photo

[0062] Customer A launches the application on the device, uploads a photo of himself, and inputs his hair volume: normal, texture: straight, desired length: short, and style preference: bob.

[0063] 2. The server analyzes the data

[0064] Using image recognition technology, the characteristics of Customer A's hair are extracted. The analysis reveals that hair volume is normal, length is medium, and color is brown.

[0065] 3. The server generates the hairstyle

[0066] Generate three bob styles based on the client's requests and hair characteristics.

[0067] 4. The server acquires trend information

[0068] We collect the latest trend information and reflect the trends in bob styles from that information.

[0069] 5. The server runs the coloring simulation.

[0070] Generates four color variations (e.g., natural brown, light brown, dark brown, reddish brown).

[0071] 6. The server sends the proposal results

[0072] Suggested hairstyles and color variations are sent to your device.

[0073] 7. The device displays the results and receives feedback

[0074] Customer A reviews the proposed styles, selects the style they like, and enters their feedback.

[0075] This allows customer A to receive suggestions for the best hairstyle that suits his or her desires.

[0076] The processing flow will be explained below.

[0077] Step 1:

[0078] When a user launches the application on their device, the login screen appears. The user enters their email address and password and taps the "Login" button.

[0079] Step 2:

[0080] The terminal sends login information to the server. The server checks the received login information against a database, and if authentication is successful, starts a session and responds to the terminal. If authentication fails, it returns an error message.

[0081] Step 3:

[0082] After successful login, the device will display the main screen and present the user with a photo upload screen where they can select a photo from their photo library or take a new photo and upload it.

[0083] Step 4:

[0084] Once the user selects or takes a photo, the device will then display a styling request input screen, where the user can enter the desired hair volume, texture, length, styling preferences, etc., and tap the "Send" button.

[0085] Step 5:

[0086] The device sends the uploaded photos and entered styling requests to the server, which receives this data and stores it in a database.

[0087] Step 6:

[0088] The server uses image recognition technology to analyze the received photo data and extract the customer's hair characteristics (length, volume, texture, color, etc.), which are then stored in a database.

[0089] Step 7:

[0090] The server uses an AI model to generate the optimal hairstyle based on the customer's request and the results of image analysis. The AI ​​model also takes into account past data and trend information when proposing a style.

[0091] Step 8:

[0092] The server uses external APIs and internal databases to collect the latest hairstyle trend information and makes additional hairstyle suggestions based on that information, which are also stored in the database.

[0093] Step 9:

[0094] The server then runs a virtual coloring simulation on the generated hairstyle, using an AI model to generate multiple color variations that match the hair color and skin tone, and stores the results.

[0095] Step 10:

[0096] The server compiles the generated hairstyle suggestions and coloring simulation results and sends them to the terminal, which receives the suggestions and displays them to the customer.

[0097] Step 11:

[0098] The user can check the proposed hairstyles and color variations on the device, select the style they like, and enter their feedback, which is then sent to the server.

[0099] Step 12:

[0100] The server stores the received feedback in a database and uses it for future suggestions. This feedback is also used as training data for the AI ​​model.

[0101] In this way, the entire system works together to suggest the best hairstyle for the user, and the quality of the service is continuously improved by incorporating feedback.

[0102] Example 1

[0103] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0104] Conventional hairstyle suggestion systems have the problem of being unable to fully customize hairstyles to reflect the user's requests and hair characteristics, resulting in low user satisfaction. They also lacked the functionality to provide coloring simulations and the latest trend information in real time. This made it difficult to suggest optimal hairstyles that met the diverse needs of users.

[0105] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0106] In this invention, the server includes means for receiving user image data, means for receiving the user's hairstyle requests, and means for analyzing the image data and extracting the user's hair characteristics. This makes it possible to propose customized hairstyles that are optimal for individual users by using a generation AI model means for generating an optimal hairstyle based on the user's requests and hair characteristics, and a means for proposing the generated hairstyle to the user.

[0107] "Image data" refers to photographs or image files showing the user's face or hair condition.

[0108] "Requests" refer to specific requests and wishes such as the user's desired hairstyle, hair volume, texture, length, etc.

[0109] "Hair characteristics" refers to attribute information such as hair length, volume, texture, and color obtained by analyzing image data.

[0110] A "generative AI model" refers to a system that uses machine learning algorithms to generate optimal hairstyles based on the user's requests and hair characteristics.

[0111] "Suggestion" refers to the act of displaying the generated hairstyle and coloring simulation results to the user and asking for their selection and feedback.

[0112] "Virtual coloring simulation" refers to a technology that virtually simulates different hair color variations for a generated hairstyle.

[0113] "Trend information" refers to information about the latest hairstyles and fashions, and is obtained from external data sources and APIs.

[0114] This invention is a system that efficiently proposes customized hairstyles desired by users using a beauty salon application. The system receives the user's image data and requests, analyzes them, and proposes optimal hairstyles using a generative AI model. It also makes additional suggestions based on coloring simulations and trend information.

[0115] Overall system configuration

[0116] The system consists of the following main components:

[0117] 1. Photo upload system (terminal): A user interface for users to upload their own images.

[0118] 2. Request input system (terminal): An interface for users to input desired styling elements and specific requests.

[0119] 3. Data analysis system (server): Technology for analyzing received image data and extracting the user's hair characteristics.

[0120] 4. Generation system (server): A generative AI model that generates optimal hairstyles based on the extracted hair characteristics and the user's requests.

[0121] 5. Proposal system (terminal): An interface for proposing generated hairstyles to users.

[0122] 6. Coloring Simulation System (Server): Technology that simulates virtual coloring.

[0123] 7. Trend information collection and suggestion system (server): A function that collects the latest trend information and suggests additional hairstyles based on it.

[0124] Hardware and software used

[0125] 1. Terminal: A device operated by a user, such as a smartphone, tablet, or PC.

[0126] 2. Server: A cloud server or dedicated server responsible for data analysis, processing of generative AI models, coloring simulation, and trend information collection.

[0127] 3. Software:

[0128] Image recognition technology: Uses libraries such as TensorFlow and OpenCV.

[0129] Generative AI models: Use machine learning algorithms such as GANs (generative adversarial networks).

[0130] External API: API for collecting external trend information (e.g. FashionTrendAPI, etc.).

[0131] Specific examples

[0132] Example 1: Proposing a new hairstyle to user A

[0133] 1. User uploads a photo

[0134] User A launches the application and uploads a photo of themselves. For example, they can take a frontal photo using their smartphone camera and import it into the app. User A enters their hair volume: normal, texture: straight, desired length: short, and style preference: bob.

[0135] 2. The server analyzes the data

[0136] The server uses image recognition technology to extract the hair characteristics of User A. Specifically, it uses the TensorFlow library to obtain analysis results such as hair length (medium), volume (normal), and color (brown).

[0137] 3. The server generates the hairstyle

[0138] Based on the analyzed hair characteristics and customer requests, three bob styles are generated using a generative AI model (e.g., GAN).

[0139] 4. The server acquires trend information

[0140] The server uses an external API (e.g., FashionTrendAPI) to obtain the latest hairstyle trend information, and reflects the trend of bob styles based on that information.

[0141] 5. The server runs the coloring simulation.

[0142] We use OpenCV to input data into a color model and generate four color variations (e.g., natural brown, light brown, dark brown, and reddish brown).

[0143] 6. The server sends the proposal results

[0144] The proposed hairstyle and color variation results are sent to the terminal and notified to the user.

[0145] 7. The device displays the results and receives feedback

[0146] User A checks the proposed styles, selects the one they like, and enters their feedback, which is then sent to the server.

[0147] Prompt Sentence Examples

[0148] "Based on the photo below, create hairstyle suggestions and color variations for a short bob. The hair texture is straight and the hair volume is medium."

[0149] This system allows users to receive suggestions for hairstyles that best suit their preferences, and also allows them to further customize their style based on a wide range of color variations and trend information.

[0150] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0151] Step 1: User Login and Authentication

[0152] When the user launches the app, the device displays the login screen. The user enters their username and password and presses the "Login" button. The server receives the username and password sent from the device and authenticates them by checking them against the database. The input data are the username and password, and the output is the success or failure of the authentication.

[0153] Step 2: Upload photos and enter your request

[0154] If authentication is successful, the device displays an upload screen. The user takes a photo of themselves or selects one from the gallery and uploads it. After the photo is uploaded, the device displays a request input screen. The user enters desired information such as hair volume, texture, length, and style. The device sends the uploaded photo and request data to the server. The input is the photo data and request data, and the output is the data sent to the server.

[0155] Step 3: Data analysis

[0156] The server stores the photo data and request data received from the device in a database. Next, the server analyzes the photo data using image recognition technology (e.g., TensorFlow) to extract the user's hair characteristics. For example, it analyzes information such as hair length, volume, texture, and color. The input is the photo data and request data, and the output is hair feature data.

[0157] Step 4: Hairstyle generation

[0158] The server inputs data into a generative AI model (e.g., GAN) based on the analyzed hair characteristics and the user's request. The model generates the optimal hairstyle and stores the results in an internal database. The input is hair characteristic data and request data, and the output is the generated hairstyle data.

[0159] Step 5: Collect trend information and make additional suggestions

[0160] The server accesses an external API (e.g., FashionTrendAPI) to collect the latest hairstyle trend information. It analyzes the collected trend information and reflects it in the generated hairstyle to make additional suggestions. The input is trend information, and the output is hairstyle data that reflects the trend.

[0161] Step 6: Coloring simulation

[0162] The server inputs data into a color model (e.g., OpenCV) for the generated hairstyle. The model generates multiple color variations and stores them in an internal database. The input is the generated hairstyle data, and the output is the color variation data.

[0163] Step 7: Submit your proposal

[0164] The server compiles the created hairstyle and color variation results and sends the proposal results in JSON format to the device. The device receives this and displays the proposal to the user. The input is hairstyle data and color variation data, and the output is the proposal result data.

[0165] Step 8: Viewing and Feedback on Proposal Results

[0166] The user checks the proposed hairstyles and color variations on the device, selects the style they like, and enters their feedback. The device then sends this feedback to the server. The input is the user's selection and feedback, and the output is the feedback data.

[0167] (Application example 1)

[0168] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0169] Conventional hair style suggestion systems for beauty salons are capable of generating hairstyles based on image analysis and customer requests, but it is difficult to propose these results to customers in real time. Furthermore, they lack support for customers to visually visualize the proposed style. Therefore, there is a need for a method that combines real-time performance and visual clarity, both of which are necessary for proposing hairstyles that satisfy customers.

[0170] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0171] In this invention, the server includes means for receiving image data of a customer, means for receiving a hairstyle request from the customer, means for analyzing the image data to extract characteristics of the customer's hair, means for generating an optimal hairstyle based on the customer's requests and hair characteristics, means for proposing the generated hairstyle to the customer, and means for displaying the generated hairstyle in real time using a wearable device such as smart glasses, thereby enabling visually easy-to-understand hairstyle suggestions and virtual coloring simulations to be provided to the customer in real time.

[0172] The "means for receiving image data of a customer" is an interface for inputting or uploading image data including a customer's facial photograph and hair characteristics in the beauty salon system.

[0173] The "means for receiving customer requests regarding hairstyle" is an interface for the customer to input or select styling requirements such as the desired hairstyle and coloring.

[0174] "Means for analyzing image data and extracting characteristics of a customer's hair" refers to algorithms or technologies for analyzing received image data of a customer and identifying characteristics such as hair length, volume, texture, and color.

[0175] The "means for generating the optimal hairstyle based on the customer's requests and hair characteristics" is an algorithm for designing and generating the optimal hairstyle using an AI model based on the extracted hair characteristics and customer requests.

[0176] The "means for proposing the generated hairstyle to the customer" is a display interface for visually presenting the hairstyle generated by the system to the customer and obtaining selection and feedback.

[0177] "Means for displaying the generated hairstyle in real time using a wearable device such as smart glasses" refers to technology that uses a wearable device such as smart glasses worn by a hairdresser to show the generated hairstyle to the customer in real time.

[0178] The "means for performing virtual coloring simulation" is a technology for applying multiple color variations to a generated hairstyle and visually simulating the results.

[0179] "Means of collecting the latest trend information and proposing additional hairstyles to customers based on the trends" is a function that collects the latest hairstyle trend information from external APIs and databases and makes additional suggestions to customers based on those trends.

[0180] The present invention provides a specific method for building a system for a beauty salon to suggest customized hairstyles to customers, which can display and suggest optimal hairstyles to customers in real time using a wearable device such as smart glasses.

[0181] 1. Hardware and Software Configuration

[0182] The system consists of the following major hardware and software components:

[0183] Hardware:

[0184] Smart glasses (e.g., Google Glass, Vuzix Blade)

[0185] Cloud Server

[0186] software:

[0187] Cloud-based data processing systems (e.g., AWS Lambda, Microsoft Azure)

[0188] Image recognition technology (e.g., OpenCV, Google Cloud Vision API)

[0189] Hairstyle generation system using AI models (e.g. TensorFlow, PyTorch)

[0190] Voice recognition technology (e.g., Google Speech-to-Text API)

[0191] 2. System Operation Overview

[0192] The system works as follows:

[0193] 1. Collection of customer image data

[0194] A hairdresser puts on smart glasses and uses the built-in camera to take a photo of the customer's face.

[0195] After the photo is taken, the image data is sent to a cloud server.

[0196] 2. Gathering customer requirements

[0197] The hairdresser uses voice recognition to input the customer's desired hairstyle and coloring requests.

[0198] The request data is also sent to the cloud server.

[0199] 3. Analysis of image data

[0200] The cloud server analyzes the received image data and extracts the customer's hair characteristics (length, volume, texture, color, etc.).

[0201] Image recognition technologies used include OpenCV and Google Cloud Vision API.

[0202] 4. Hairstyle generation

[0203] Based on the analyzed hair characteristics and the customer's requests, an AI model is used to generate the optimal hairstyle.

[0204] The AI ​​models used include TensorFlow and PyTorch.

[0205] 5. Virtual Coloring Simulation

[0206] A virtual coloring simulation is performed by applying multiple color variations to the generated hairstyle.

[0207] The simulation results are also stored on the cloud server.

[0208] 6. Real-time display of results

[0209] The generated hairstyle and color variation results are sent to the smart glasses.

[0210] Hairdressers make real-time suggestions to customers through smart glasses.

[0211] Specific examples

[0212] Example 1: Proposing a hairstyle for customer A

[0213] 1. The hairdresser puts on the smart glasses and takes a photo of Customer A's face.

[0214] 2. Using the voice recognition function, Customer A inputs the style he or she desires (e.g., short bob, brown color).

[0215] 3. The captured image data and requested data are sent to the cloud server.

[0216] 4. The image data is analyzed on the cloud server, and hair characteristics (e.g., medium length, straight, black hair) are extracted.

[0217] 5. Based on the analysis results and your request, the AI ​​model will generate three short bob styles and perform a virtual coloring simulation.

[0218] 6. The results are sent to the smart glasses, and the hairdresser makes suggestions to Customer A through the glasses.

[0219] 7. Customer A chooses the best style and enters feedback via voice recognition.

[0220] Prompt Sentence Examples

[0221] To ask an AI model to generate a hairstyle based on a customer's request, use the following prompt:

[0222] "Generate a suitable hairstyle based on the customer's facial photo and requests."

[0223] This process allows customers to visually see in real time which hairstyle best suits them, ensuring a highly satisfying service.

[0224] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0225] Step 1:

[0226] Input: A customer comes in and the hairdresser puts on the smart glasses.

[0227] What it does: A hairdresser uses the built-in camera in the smart glasses to take a photo of the customer's face.

[0228] Output: The captured image data is stored in the smart glasses.

[0229] Step 2:

[0230] Input: A photo of the customer's face.

[0231] Specific operation: A photo of the customer's face is sent from the smart glasses to a cloud server.

[0232] Output: The cloud server receives and stores the facial photo data.

[0233] Step 3:

[0234] Input: Customer's hairstyle and coloring requests.

[0235] Specific operation: The hairdresser uses the voice recognition function to input the customer's request by voice.

[0236] Output: The audio data is sent to a cloud server and converted into text data.

[0237] Step 4:

[0238] Input: Customer's photo data and request data.

[0239] How it works: The cloud server uses image recognition technology (e.g., OpenCV or Google Cloud Vision API) to analyze the customer's hair characteristics (length, volume, texture, color, etc.).

[0240] Output: Analysis data containing hair features is generated and saved.

[0241] Step 5:

[0242] Input: Customer's hair characteristics analysis data and request data.

[0243] What it does: A cloud server uses an AI model (e.g., TensorFlow or PyTorch) to generate the optimal hairstyle.

[0244] Output: The generated hairstyle data is saved.

[0245] Step 6:

[0246] Input: Generated hairstyle data.

[0247] Specific operation: The cloud server performs a virtual coloring simulation and applies multiple color variations.

[0248] Output: Hairstyle data including coloring simulation results is generated and saved.

[0249] Step 7:

[0250] Input: Hairstyle data including coloring simulation results.

[0251] Specific operation: The cloud server sends hairstyle data to the smart glasses.

[0252] Output: The transmitted hairstyle data is displayed on the smart glasses.

[0253] Step 8:

[0254] Input: Hairstyle data displayed on smart glasses.

[0255] How it works: A hairdresser uses smart glasses to suggest the best hairstyle and color options to a customer in real time.

[0256] Output: The customer reviews the suggested hairstyles and selects the best one.

[0257] Step 9:

[0258] Input: Customer selection and feedback.

[0259] Specific operation: The hairdresser inputs the customer's selection and feedback through voice recognition in the smart glasses.

[0260] Output: The feedback data is sent to the cloud server and stored.

[0261] This allows the server and terminal to work together, making it possible to suggest the best hairstyle for each customer in real time.

[0262] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0263] This invention is an application system for beauty salons that proposes hairstyles customized to meet customer needs, and provides more personalized services by combining it with an emotion engine that recognizes the user's emotions. This system analyzes the customer's image data, generates hairstyles according to their requests, and makes suggestions based on coloring simulations and trend information. The emotion engine also recognizes the user's emotions and optimizes the suggestions based on these.

[0264] Overall system configuration

[0265] The system consists of the following main components:

[0266] 1. Photo upload system (terminal): A user interface for customers to upload their own images.

[0267] 2. Request input system (terminal): An interface for customers to input desired styling elements and specific requests.

[0268] 3. Data analysis system (server): Technology for analyzing received image data and extracting the characteristics of the customer's hair.

[0269] 4. Generation system (server): An algorithm for generating the optimal hairstyle based on the extracted hair characteristics and customer requests.

[0270] 5. Proposal system (terminal): An interface for proposing the generated hairstyle to the customer.

[0271] 6. Coloring Simulation System (Server): Technology that simulates virtual coloring.

[0272] 7. Trend information collection and suggestion system (server): A function that collects the latest trend information and suggests additional hairstyles based on it.

[0273] 8. Emotion Engine (Server): Technology that recognizes user emotions and adjusts suggestions based on them.

[0274] Program Processing Overview

[0275] 1. Upload photos and input requests (device processing)

[0276] After launching the application and logging in, the user is presented with a photo upload screen. The user can select a photo from their photo library or take a new photo and upload it. A styling request input screen is then displayed, where the user enters information such as the desired hair volume, texture, length, and style, and the data is sent to the server.

[0277] 2. Data analysis (server processing)

[0278] The server stores the photo data and requests received from the device in a database, and then uses image recognition technology to extract the customer's hair characteristics, such as hair length, volume, texture, and color.

[0279] 3. Hairstyle generation (server processing)

[0280] The server uses an AI model to generate the optimal hairstyle based on the analyzed hair characteristics and the customer's requests, and the generated style is stored in an internal database.

[0281] 4. Collecting trend information and making additional suggestions (server processing)

[0282] The server uses external APIs and internal databases to collect the latest hairstyle trend information and provide additional hairstyle suggestions based on that information, which are also stored in the database.

[0283] 5. Coloring simulation (server processing)

[0284] The server performs a virtual coloring simulation on the generated hairstyle, using an AI model to generate and store multiple color variations that match different hair colors and skin tones.

[0285] 6. Emotion Recognition and Suggestion Adjustment (Server Processing)

[0286] The server uses an emotion engine to analyze the facial expressions and voice of the user reviewing the proposed hairstyle and recognize their emotions. Based on the recognized emotions, the server adjusts the proposal and provides personalized suggestions.

[0287] 7. Sending the proposal results (server processing)

[0288] The server compiles the generated hairstyle suggestions, coloring simulation results, and suggestions adjusted by emotion recognition, and sends them to the terminal, which receives the suggestions and displays them to the customer.

[0289] 8. Displaying and Feedback of Proposal Results (Device Processing)

[0290] The user can check the proposed hairstyles and color variations on the device, select the style they like, and enter their feedback, which is then sent to the server.

[0291] 9. Saving feedback and reflecting it in the next proposal (server processing)

[0292] The server stores the received feedback in a database and uses it for future suggestions. This feedback is also used as training data for the AI ​​model, and is used to increase user satisfaction.

[0293] Specific examples

[0294] Example 1: Proposing a new hairstyle for Customer B

[0295] 1. User uploads a photo

[0296] Customer B launches the application on the device, uploads a photo of himself, and inputs his styling requests: hair volume: thick, texture: wavy, desired length: medium, style preference: casual.

[0297] 2. The server analyzes the data

[0298] Using image recognition technology, the characteristics of Customer B's hair are extracted. The analysis reveals that the hair volume is thick, the length is medium, and the color is black.

[0299] 3. The server generates the hairstyle

[0300] Generate three casual styles based on the customer's requests and hair characteristics.

[0301] 4. The server acquires trend information

[0302] We collect the latest trend information and reflect the latest trends in casual style.

[0303] 5. The server runs the coloring simulation.

[0304] Generates four different color variations (e.g., natural black, dark brown, light brown, reddish brown).

[0305] 6. Emotion recognition and suggestion adjustment

[0306] As the user reviews the suggested styles, an emotion engine analyzes the user's facial expressions. For example, if surprise or joy is detected, the server will adjust the suggestions based on that emotion and make other suggestions as well.

[0307] 7. The server sends the proposal results

[0308] Suggested hairstyles and color variations, tailored suggestions based on emotions, are sent to the device.

[0309] 8. The device displays the results and receives feedback

[0310] Customer B reviews the proposed styles, selects the style they like, and provides feedback.

[0311] 9. The server stores the feedback

[0312] Receive feedback and incorporate it into your next proposal.

[0313] This allows Customer B to not only receive suggestions for the optimal hairstyle and coloring simulation that suits his or her preferences, but also receive more personalized suggestions based on his or her emotions, resulting in high levels of satisfaction.

[0314] The processing flow will be explained below.

[0315] Step 1:

[0316] When a user launches the application on their device, the login screen appears. The user enters their email address and password and taps the "Login" button.

[0317] Step 2:

[0318] The terminal sends login information to the server. The server checks the received login information against a database, and if authentication is successful, starts a session and responds to the terminal. If authentication fails, it returns an error message.

[0319] Step 3:

[0320] After successful login, the device will display the main screen and present the user with a photo upload screen where they can select a photo from their photo library or take a new photo and upload it.

[0321] Step 4:

[0322] Once the user selects or takes a photo, the device will then display a styling request input screen, where the user can enter the desired hair volume, texture, length, styling preferences, etc., and tap the "Send" button.

[0323] Step 5:

[0324] The device sends the uploaded photos and entered styling requests to the server, which receives this data and stores it in a database.

[0325] Step 6:

[0326] The server uses image recognition technology to analyze the received photo data and extract the customer's hair characteristics (length, volume, texture, color, etc.), which are then stored in a database.

[0327] Step 7:

[0328] The server uses an AI model to generate the optimal hairstyle based on the customer's request and the results of image analysis. The AI ​​model also takes into account past data and trend information when proposing a style.

[0329] Step 8:

[0330] The server uses external APIs and internal databases to collect the latest hairstyle trend information and makes additional hairstyle suggestions based on that information, which are also stored in the database.

[0331] Step 9:

[0332] The server then runs a virtual coloring simulation on the generated hairstyle, using an AI model to generate multiple color variations that match the hair color and skin tone, and stores the results.

[0333] Step 10:

[0334] The server uses an emotion engine to analyze the facial expressions and voice of the user confirming the proposed hairstyle and recognize their emotions. For example, emotions such as joy or dissatisfaction can be extracted from the user's facial expressions and voice.

[0335] Step 11:

[0336] The server then tailors the suggestions based on the recognized emotion, for example, by prioritizing hairstyles that indicate a happy emotion to the user.

[0337] Step 12:

[0338] The server sends the generated hairstyle suggestions, coloring simulation results, and suggestions adjusted by emotion recognition to the terminal, which receives the suggestions and displays them to the customer.

[0339] Step 13:

[0340] The user can check the proposed hairstyles and color variations on the device, select the style they like, and enter their feedback, which is then sent to the server.

[0341] Step 14:

[0342] The server stores the received feedback in a database and uses it for future suggestions. This feedback is also used as training data for the AI ​​model.

[0343] In this way, the entire system works together to suggest the best hairstyle for the user and improve the quality of service based on emotion recognition and feedback.

[0344] Example 2

[0345] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0346] While existing hair salon application systems suggest hairstyles based on customer requests, they lack personalized suggestions that take into account customer emotions and trend information. Furthermore, there is no system in place to incorporate feedback on suggested styles into future suggestions, which means customer satisfaction is not fully improved. Furthermore, coloring simulations are limited, making it difficult to fully address the diverse needs of customers.

[0347] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0348] In this invention, the server includes a means for receiving image data of a customer, a means for receiving the customer's hairstyle requests, and a means for analyzing the image data to extract the customer's hair characteristics. This makes it possible to generate and propose an optimal hairstyle based on the customer's requests and hair characteristics. The server also includes a means for recognizing the user's emotions and optimizing the proposal based on the emotions, a means for receiving feedback from the customer and reflecting it in future proposals, a means for performing a virtual coloring simulation for the generated hairstyle, and a means for collecting the latest trend information and proposing additional hairstyles based on the trends to the customer. This makes it possible to propose hairstyles that are personalized to the customer, thereby increasing customer satisfaction.

[0349] The "means for receiving image data from customers" is a function for acquiring images uploaded by customers, converting them into a format that can be used within the system, and saving them.

[0350] The "means for receiving customer requests regarding hairstyle" is a function for inputting information regarding the hairstyle desired by the customer, and collecting and saving the information.

[0351] The "means for analyzing image data and extracting characteristics of the customer's hair" is a function for identifying and extracting characteristics such as hair length, volume, texture, and color based on the received image data.

[0352] "A means for generating the optimal hairstyle based on the customer's requests and hair characteristics" is a function that uses an AI model to generate multiple candidate hairstyles based on the customer's requests and the results of data analysis.

[0353] The "means for proposing generated hairstyles to a customer" is a function for presenting generated hairstyles to a customer and encouraging them to select and evaluate them.

[0354] "Means for recognizing the user's emotions and optimizing suggestions based on them" refers to a function that analyzes the user's facial expressions and voice data and adjusts the suggestions based on their emotional state.

[0355] "Means of receiving feedback from customers and reflecting it in future proposals" is a function that records the evaluations and comments that customers make on proposals and uses them to help with future proposals.

[0356] The "means for performing a virtual coloring simulation for the generated hairstyle" is a function for simulating different hair colors for the generated hairstyle and presenting them to the customer.

[0357] "Means for collecting the latest trend information and proposing additional hairstyles based on the trends to customers" refers to a function for obtaining the latest hairstyle trend data from an external source and proposing additional hairstyles based on that data to customers.

[0358] This invention is an application system for beauty salons that proposes hairstyles customized to the customer's needs and provides more personalized services by combining it with an emotion engine that recognizes the user's emotions. The system consists of the following main components:

[0359] 1. Photo upload system (terminal)

[0360] This is a user interface for customers to upload their own images. Users launch the app on their smartphone, log in, and then select a photo from their photo library or take a new photo and upload it.

[0361] 2. Request input system (terminal)

[0362] This is the interface where the customer inputs the styling elements and specific requests they want, such as hair volume, texture, length, and style, and then sends the data to the server.

[0363] 3. Data analysis system (server)

[0364] This technology analyzes the received image data and extracts the customer's hair characteristics. This process uses the image recognition library OpenCV and the deep learning framework TensorFlow. For example, features such as hair length, volume, texture, and color can be extracted from the image.

[0365] 4. Generation System (Server)

[0366] Based on the analyzed hair characteristics and the customer's requests, an optimal hairstyle is generated using an AI model (e.g., a generative artificial network (GAN)). The generated style is saved in an internal database. An example of a prompt sentence is, "The client has a lot of hair, the texture is wavy, the desired length is medium, and the style preference is casual."

[0367] 5. Proposed system (terminal)

[0368] This is an interface for proposing generated hairstyles to customers. Users can check multiple hairstyles generated on the app and enter feedback. The feedback is sent to the server.

[0369] 6. Coloring Simulation System (Server)

[0370] A virtual coloring simulation is performed on the generated hairstyle, using an AI model to generate multiple color variations (e.g., natural black, dark brown, light brown, reddish brown) that match the hair color and skin tone, and then stored in a database.

[0371] 7. Trend information collection and proposal system (server)

[0372] This function collects the latest trend information from external APIs (e.g., SNS trend APIs) and internal databases, and then suggests additional hairstyles based on that information. Trend information is updated regularly, and new styles are suggested to customers.

[0373] 8. Emotion Engine (Server)

[0374] This technology recognizes the user's emotions and optimizes suggestions based on them. It uses the Emotion API to analyze the user's facial expressions and voice to recognize their emotional state. For example, if surprise or joy is detected, the system reevaluates the suggestions and makes personalized suggestions.

[0375] 9. Feedback Collection System (Server)

[0376] This function receives customer feedback on the proposed style and reflects it in future proposals. The received feedback is stored in a database and used as learning data for the AI ​​model.

[0377] This allows the system to not only propose optimal hairstyles based on the customer's requests and hair characteristics, but also provide personalized services that take into account the user's emotions and trend information.Furthermore, by incorporating feedback into the next proposal, customer satisfaction can be further increased.

[0378] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0379] Step 1:

[0380] User

[0381] The user launches the application and logs in. After logging in, a photo upload screen is displayed. The user selects a photo from the photo library or takes a new photo using the device's camera. The input is the user's photo data, and the output is the uploaded image data. Specifically, the user taps the smartphone app to launch it, and then enters their username and password for authentication. After authentication, the user selects or takes a photo and sends it to the server.

[0382] Step 2:

[0383] server

[0384] The server receives the uploaded image data and stores it in a database. It also analyzes it for the next step. The input is the uploaded image data, and the output is the image data stored in the database. Specifically, it receives an HTTP request and stores the image data in the database.

[0385] Step 3:

[0386] User

[0387] The user moves to the styling request input screen, enters information such as hair volume, texture, length, and style, and sends it to the server. The input is the request data entered by the user, and the output is the request data sent. Specifically, the user selects hair characteristics on the request input screen and taps the "Send" button.

[0388] Step 4:

[0389] server

[0390] The server stores the received request data in a database and analyzes it together with the image data. The input is the image data and the request data, and the output is the analyzed hair feature data. This analysis is performed using OpenCV and TensorFlow. Specifically, an image recognition algorithm is used to identify features such as hair length, volume, texture, and color.

[0391] Step 5:

[0392] server

[0393] The server uses an AI model (e.g., GAN) to generate the optimal hairstyle based on the analyzed hair features and desired data. The input is the analyzed hair feature data and desired data, and the output is the generated hairstyle data. Specifically, the server inputs a prompt statement into the AI ​​model to generate multiple hairstyles.

[0394] Step 6:

[0395] server

[0396] The server saves the generated hairstyle data in a database and sends it to the device. The input is the generated hairstyle data, and the output is the hairstyle suggestion data sent to the device. The specific operation is to save the results in a database and send them to the device as an HTTP response.

[0397] Step 7:

[0398] User

[0399] The user reviews the hairstyle suggestions generated on the device and enters feedback. The input is the proposed hairstyle data, and the output is the feedback data. Specifically, the user browses the suggested hairstyles on the app, selects them, and enters comments.

[0400] Step 8:

[0401] server

[0402] The server stores the received feedback data in a database and reflects it in the next proposal. The input is the feedback data, and the output is an updated database. Specifically, the server analyzes the feedback information and updates the database to use it in the next proposal.

[0403] Step 9:

[0404] server

[0405] The server uses an external API to collect the latest trend information. The input is the trend information obtained from the external API, and the output is the trend information stored in the database. Specifically, the server periodically sends an API request to obtain and store the latest trend information data.

[0406] Step 10:

[0407] server

[0408] The server performs a virtual coloring simulation for the generated hairstyle. The input is the generated hairstyle data and the user's skin tone data, and the output is multiple color variation data. Specifically, the server uses the coloring simulation model to generate different color variations of the hairstyle and save them in a database.

[0409] Step 11:

[0410] server

[0411] The server analyzes the user's facial expressions and voice to recognize emotions. The input is the user's facial expression and voice data, and the output is the recognized emotion data. Specifically, it calls the Emotion API, analyzes the user's facial expression and voice data, and identifies the emotion.

[0412] Step 12:

[0413] server

[0414] The server optimizes the suggestions based on the recognized emotion. The input is the recognized emotion data and the suggestion data, and the output is the suggestion data adjusted based on the emotion. Specifically, the server uses the AI ​​model to reevaluate the suggestions and reconfigure the optimal hairstyle based on the emotion.

[0415] (Application example 2)

[0416] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0417] Conventional hair salon application systems suggest hairstyles based on customer requests, but do not provide personalized suggestions based on the customer's emotions, making it difficult to maximize customer satisfaction. In addition to simply suggesting hairstyles, salons are also required to provide coloring simulations and trend information, but there is a lack of systems that provide these functions in an integrated manner. Furthermore, there is a need for a system that can incorporate real-time emotion recognition to make optimal suggestions based on the customer's emotions.

[0418] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0419] In this invention, the server includes a means for receiving image data of a customer, a means for receiving the customer's hairstyle requests, a means for analyzing the image data to extract the customer's hair characteristics, a means for generating an optimal hairstyle based on the customer's requests and hair characteristics, a means for proposing the generated hairstyle to the customer, and a means for analyzing the customer's facial expressions and voice to recognize emotions and adjust the proposed result. This makes it possible to propose an optimal hairstyle based on the customer's emotions in real time, in addition to their requests. It is also possible to perform coloring simulations and propose additional hairstyles based on the latest trend information, which can significantly improve overall customer satisfaction.

[0420] The "means for receiving customer image data" is a device that provides an interface for users to upload their photos to the system.

[0421] The "means for receiving customer requests regarding hairstyle" is a device that provides an interface through which the user can input information regarding the hairstyle and coloring they desire.

[0422] The "means for analyzing image data to extract characteristics of the customer's hair" is a device that includes an algorithm that analyzes uploaded image data and identifies characteristics such as hair length, texture, and color.

[0423] The "means for generating the optimal hairstyle based on the customer's requests and hair characteristics" is a device that integrates the user's requests with analyzed hair characteristics and generates the optimal hairstyle using an AI model.

[0424] The "means for proposing generated hairstyles to a customer" is a device that presents the hairstyles generated by the system to the user and provides an interface for receiving selections and feedback.

[0425] The "means for analyzing the customer's facial expressions and voice to recognize emotions and adjust the suggested results" refers to a device that includes an algorithm that analyzes the user's facial expressions and voice, recognizes emotions in real time, and adjusts the suggested hairstyle based on the results.

[0426] This invention relates to a system for improving customer experience in beauty salons. Specifically, the system receives customer image data and requests, analyzes them to suggest hairstyles and colorings, and recognizes emotions to adjust the suggestions, providing a more personalized service.

[0427] The system includes, among other things, the following main components:

[0428] 1. Means of receiving customer image data

[0429] It is mainly implemented in devices such as smartphones and tablets, and provides an interface for users to upload their own photos. For example, a smartphone application allows a user to take a selfie with their camera and upload it to the system. This photo data is then sent to a cloud server.

[0430] 2. A means of receiving customer requests regarding hairstyles

[0431] It provides an interface for users to input their desired hairstyle and coloring information. For example, users can input detailed requests such as hair length, texture, color, and style preferences into the app.

[0432] 3. A method for analyzing image data to extract the characteristics of a customer's hair

[0433] Using image recognition technology, the uploaded photo is analyzed to extract features such as hair length, texture, and color. This analysis can be performed using libraries such as TensorFlow and OpenCV. The analysis results are used in the next step.

[0434] 4. A means to generate optimal hairstyles based on customer requests and hair characteristics

[0435] Using a generative AI model, we generate the optimal hairstyle based on the user's request and analyzed hair characteristics. We use deep learning frameworks such as Keras and PyTorch to propose several hairstyle candidates.

[0436] 5. A means of proposing generated hairstyles to customers

[0437] The proposed hairstyles are presented to the user and an interface is provided for selection and feedback. For example, the generated hairstyles are displayed on the app, and the user can browse them and select the style they prefer.

[0438] 6. A method to recognize emotions by analyzing customer facial expressions and voice and adjust the proposal results

[0439] As users review the suggested hairstyles, the system analyzes their facial expressions and voice to recognize their emotions. This process is carried out using emotion recognition software such as EmotionEngine. Based on the recognized emotions, the system adjusts the suggestions to improve user satisfaction.

[0440] Specific examples

[0441] Let's take a specific example where customer C is looking for a new hairstyle at a hair salon.

[0442] 1. Upload a photo

[0443] Customer C opens the app, takes a photo of themselves, and uploads it.

[0444] 2. Input your request

[0445] Hair length: Medium

[0446] Hair Texture:Wave

[0447] Favorite color: Dark

[0448] 3. Data Analysis

[0449] It is analyzed as follows: Length: Medium, Texture: Wavy, Color: Black.

[0450] 4. Hairstyle Generation

[0451] Based on the analysis results, several new styles are generated.

[0452] 5. Emotion recognition

[0453] Recognize the happy expression on Customer C's face when he sees the proposed style.

[0454] 6. Final proposal result

[0455] The tailored suggestions are displayed to Customer C on her smartphone, and she chooses the style she likes best.

[0456] Prompt Sentence Examples

[0457] Image file: "path_to_your_image_file.jpg"

[0458] Customer Request:

[0459] Hair Length: Medium

[0460] Hair Texture: Wavy

[0461] Favorite color: Dark

[0462] Suggested style: "Curl"

[0463] Emotion recognition result: "Joy"

[0464] Make your final offer based on emotion.

[0465] This allows users to not only receive suggestions for the best hairstyle and coloring simulation based on their needs, but also receive more personalized suggestions based on their emotions, providing a highly satisfying experience.

[0466] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0467] Step 1:

[0468] Upload a photo

[0469] Subject: User

[0470] Description: A user launches the application, takes a photo of themselves, and uploads it to the system. The input is image data taken with the smartphone camera, and the output is data in an image file format sent to the cloud server. Specifically, the user presses the "Upload Photo" button, selects a photo, and uploads it.

[0471] Step 2:

[0472] Request input

[0473] Subject: User

[0474] Description: A user uses an application interface to input detailed information about their desired hairstyle and color. The input is the user's style preferences provided through text and options, and the output is data about those preferences sent to the server. Specific operations include selecting length, texture, color, etc. on the "Enter Hairstyle Preferences" screen and submitting it.

[0475] Step 3:

[0476] Data analysis

[0477] Subject: Server

[0478] Description: The server analyzes and processes the image data received and extracts the user's hair features. The input is the uploaded image data, and the output is the analysis results such as hair length, texture, and color. Specifically, it uses TensorFlow and OpenCV to detect hair areas from the image and extracts their features as numerical data.

[0479] Step 4:

[0480] Hairstyle generation

[0481] Subject: Server

[0482] Description: The server uses a generative AI model to generate the optimal hairstyle based on the analyzed hair features and the user's requests. The input is hair feature data and user request data, and the output is generated hairstyle candidates. Specifically, it applies a trained model using Keras or PyTorch to generate several optimal hairstyles.

[0483] Step 5:

[0484] Providing proposal results

[0485] Subject: Server

[0486] Description: The generated hairstyle is proposed to the user and made available for viewing. The input is the data of the generated hairstyle, and the output is the proposal information sent to the user's device. Specifically, the application generates images of the multiple generated hairstyles and displays them on the application screen.

[0487] Step 6:

[0488] emotion recognition

[0489] Subject: Server

[0490] Description: The server analyzes the user's facial expressions and voice to recognize emotions. The input is the user's facial expression and voice data, and the output is the recognized emotion data. Specifically, it uses the Emotion Engine to analyze data obtained from the camera and microphone and quantifies the user's emotions.

[0491] Step 7:

[0492] Adjustment of proposed results

[0493] Subject: Server

[0494] Description: Adjusts suggested hairstyles based on emotion recognition results. The input is the recognized emotion data and initial hairstyle suggestion data, and the output is the adjusted final suggestion data. Specific operations include modifying suggested hairstyles or making additional suggestions based on the recognized emotion.

[0495] Step 8:

[0496] Provision of final proposal results

[0497] Subject: Server

[0498] Description: Provides the user with a final hairstyle proposal after adjustments. The input is the adjusted proposal data, and the output is the final proposal information sent to the user's device. Specific operations include displaying the adjusted hairstyle proposal in the application so that the user can confirm it.

[0499] Step 9:

[0500] Collecting and storing feedback

[0501] Subject: User and Server

[0502] Description: The user reviews the suggested styles, selects the one they like, and provides feedback. This feedback is sent to the server and saved. The input is the hairstyle selected by the user and the feedback data, and the output is the feedback data saved on the server. Specifically, when the user presses the "Send Feedback" button, the information entered is sent to the server and used for the next suggestion.

[0503] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0504] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0505] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0506] [Second embodiment]

[0507] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0508] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0509] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0510] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0511] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0512] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0513] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0514] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0515] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0516] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0517] In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0518] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0519] This invention is a system for building an application for beauty salons that proposes customized hairstyles that meet customer needs. This system analyzes the customer's image data, generates the optimal hairstyle, and provides coloring simulations and the latest trend information.

[0520] Overall system configuration

[0521] The system consists of the following main components:

[0522] 1. Photo upload system (terminal): A user interface for customers to upload their own images.

[0523] 2. Request input system (terminal): An interface for customers to input desired styling elements and specific requests.

[0524] 3. Data analysis system (server): Technology for analyzing received image data and extracting the characteristics of the customer's hair.

[0525] 4. Generation system (server): An algorithm for generating the optimal hairstyle based on the extracted hair characteristics and customer requests.

[0526] 5. Proposal system (terminal): An interface for proposing the generated hairstyle to the customer.

[0527] 6. Coloring Simulation System (Server): Technology that simulates virtual coloring.

[0528] 7. Trend information collection and suggestion system (server): A function that collects the latest trend information and suggests additional hairstyles based on it.

[0529] Program Processing Overview

[0530] 1. Upload photos and input requests (device processing)

[0531] The customer launches the application and logs in. After logging in, a photo upload screen appears, where the customer can take or select a photo of themselves. Next, a screen appears where the customer can enter information such as the amount, texture, length, and style of their desired hair. This data is then sent from the device to the server.

[0532] 2. Data analysis (server processing)

[0533] The server stores the photo data and requests received from the device in a database, and then uses image recognition technology to extract the customer's hair characteristics, such as hair length, volume, texture, and color.

[0534] 3. Hairstyle generation (server processing)

[0535] The server uses an AI model to generate the optimal hairstyle based on the analyzed hair characteristics and the customer's requests, and the generated style is stored in an internal database.

[0536] 4. Collecting trend information and making additional suggestions (server processing)

[0537] The server uses external APIs and internal databases to collect the latest hairstyle trends, and then provides additional hairstyle suggestions based on these trends.

[0538] 5. Coloring simulation (server processing)

[0539] The server then performs a virtual coloring simulation on the generated hairstyle, using an AI model to generate and save color variations that match the hair color and skin tone.

[0540] 6. Sending the proposal results (server processing)

[0541] The server sends the hairstyle proposal results and coloring simulation results to the terminal, which receives them and displays them to the customer.

[0542] 7. Displaying and Feedback of Proposal Results (Device Processing)

[0543] The customer checks the proposed hairstyles and color variations on the terminal and inputs their selection or feedback, which is then sent to the server.

[0544] Specific examples

[0545] Example 1: Proposing a new hairstyle to Customer A

[0546] 1. User uploads a photo

[0547] Customer A launches the application on the device, uploads a photo of himself, and inputs his hair volume: normal, texture: straight, desired length: short, and style preference: bob.

[0548] 2. The server analyzes the data

[0549] Using image recognition technology, the characteristics of Customer A's hair are extracted. The analysis reveals that hair volume is normal, length is medium, and color is brown.

[0550] 3. The server generates the hairstyle

[0551] Generate three bob styles based on the client's requests and hair characteristics.

[0552] 4. The server acquires trend information

[0553] We collect the latest trend information and reflect the trends in bob styles from that information.

[0554] 5. The server runs the coloring simulation.

[0555] Generates four color variations (e.g., natural brown, light brown, dark brown, reddish brown).

[0556] 6. The server sends the proposal results

[0557] Suggested hairstyles and color variations are sent to your device.

[0558] 7. The device displays the results and receives feedback

[0559] Customer A reviews the proposed styles, selects the style they like, and enters their feedback.

[0560] This allows customer A to receive suggestions for the best hairstyle that suits his or her desires.

[0561] The processing flow will be explained below.

[0562] Step 1:

[0563] When a user launches the application on their device, the login screen appears. The user enters their email address and password and taps the "Login" button.

[0564] Step 2:

[0565] The terminal sends login information to the server. The server checks the received login information against a database, and if authentication is successful, starts a session and responds to the terminal. If authentication fails, it returns an error message.

[0566] Step 3:

[0567] After successful login, the device will display the main screen and present the user with a photo upload screen where they can select a photo from their photo library or take a new photo and upload it.

[0568] Step 4:

[0569] Once the user selects or takes a photo, the device will then display a styling request input screen, where the user can enter the desired hair volume, texture, length, styling preferences, etc., and tap the "Send" button.

[0570] Step 5:

[0571] The device sends the uploaded photos and entered styling requests to the server, which receives this data and stores it in a database.

[0572] Step 6:

[0573] The server uses image recognition technology to analyze the received photo data and extract the customer's hair characteristics (length, volume, texture, color, etc.), which are then stored in a database.

[0574] Step 7:

[0575] The server uses an AI model to generate the optimal hairstyle based on the customer's request and the results of image analysis. The AI ​​model also takes into account past data and trend information when proposing a style.

[0576] Step 8:

[0577] The server uses external APIs and internal databases to collect the latest hairstyle trend information and makes additional hairstyle suggestions based on that information, which are also stored in the database.

[0578] Step 9:

[0579] The server then runs a virtual coloring simulation on the generated hairstyle, using an AI model to generate multiple color variations that match the hair color and skin tone, and stores the results.

[0580] Step 10:

[0581] The server compiles the generated hairstyle suggestions and coloring simulation results and sends them to the terminal, which receives the suggestions and displays them to the customer.

[0582] Step 11:

[0583] The user can check the proposed hairstyles and color variations on the device, select the style they like, and enter their feedback, which is then sent to the server.

[0584] Step 12:

[0585] The server stores the received feedback in a database and uses it for future suggestions. This feedback is also used as training data for the AI ​​model.

[0586] In this way, the entire system works together to suggest the best hairstyle for the user, and the quality of the service is continuously improved by incorporating feedback.

[0587] Example 1

[0588] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0589] Conventional hairstyle suggestion systems have the problem of being unable to fully customize hairstyles to reflect the user's requests and hair characteristics, resulting in low user satisfaction. They also lacked the functionality to provide coloring simulations and the latest trend information in real time. This made it difficult to suggest optimal hairstyles that met the diverse needs of users.

[0590] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0591] In this invention, the server includes means for receiving user image data, means for receiving the user's hairstyle requests, and means for analyzing the image data and extracting the user's hair characteristics. This makes it possible to propose customized hairstyles that are optimal for individual users by using a generation AI model means for generating an optimal hairstyle based on the user's requests and hair characteristics, and a means for proposing the generated hairstyle to the user.

[0592] "Image data" refers to photographs or image files showing the user's face or hair condition.

[0593] "Requests" refer to specific requests and wishes such as the user's desired hairstyle, hair volume, texture, length, etc.

[0594] "Hair characteristics" refers to attribute information such as hair length, volume, texture, and color obtained by analyzing image data.

[0595] A "generative AI model" refers to a system that uses machine learning algorithms to generate optimal hairstyles based on the user's requests and hair characteristics.

[0596] "Suggestion" refers to the act of displaying the generated hairstyle and coloring simulation results to the user and asking for their selection and feedback.

[0597] "Virtual coloring simulation" refers to a technology that virtually simulates different hair color variations for a generated hairstyle.

[0598] "Trend information" refers to information about the latest hairstyles and fashions, and is obtained from external data sources and APIs.

[0599] This invention is a system that efficiently proposes customized hairstyles desired by users using a beauty salon application. The system receives the user's image data and requests, analyzes them, and proposes optimal hairstyles using a generative AI model. It also makes additional suggestions based on coloring simulations and trend information.

[0600] Overall system configuration

[0601] The system consists of the following main components:

[0602] 1. Photo upload system (terminal): A user interface for users to upload their own images.

[0603] 2. Request input system (terminal): An interface for users to input desired styling elements and specific requests.

[0604] 3. Data analysis system (server): Technology for analyzing received image data and extracting the user's hair characteristics.

[0605] 4. Generation system (server): A generative AI model that generates optimal hairstyles based on the extracted hair characteristics and the user's requests.

[0606] 5. Proposal system (terminal): An interface for proposing generated hairstyles to users.

[0607] 6. Coloring Simulation System (Server): Technology that simulates virtual coloring.

[0608] 7. Trend information collection and suggestion system (server): A function that collects the latest trend information and suggests additional hairstyles based on it.

[0609] Hardware and software used

[0610] 1. Terminal: A device operated by a user, such as a smartphone, tablet, or PC.

[0611] 2. Server: A cloud server or dedicated server responsible for data analysis, processing of generative AI models, coloring simulation, and trend information collection.

[0612] 3. Software:

[0613] Image recognition technology: Uses libraries such as TensorFlow and OpenCV.

[0614] Generative AI models: Use machine learning algorithms such as GANs (generative adversarial networks).

[0615] External API: API for collecting external trend information (e.g. FashionTrendAPI, etc.).

[0616] Specific examples

[0617] Example 1: Proposing a new hairstyle to user A

[0618] 1. User uploads a photo

[0619] User A launches the application and uploads a photo of themselves. For example, they can take a frontal photo using their smartphone camera and import it into the app. User A enters their hair volume: normal, texture: straight, desired length: short, and style preference: bob.

[0620] 2. The server analyzes the data

[0621] The server uses image recognition technology to extract the hair characteristics of User A. Specifically, it uses the TensorFlow library to obtain analysis results such as hair length (medium), volume (normal), and color (brown).

[0622] 3. The server generates the hairstyle

[0623] Based on the analyzed hair characteristics and customer requests, three bob styles are generated using a generative AI model (e.g., GAN).

[0624] 4. The server acquires trend information

[0625] The server uses an external API (e.g., FashionTrendAPI) to obtain the latest hairstyle trend information, and reflects the trend of bob styles based on that information.

[0626] 5. The server runs the coloring simulation.

[0627] We use OpenCV to input data into a color model and generate four color variations (e.g., natural brown, light brown, dark brown, and reddish brown).

[0628] 6. The server sends the proposal results

[0629] The proposed hairstyle and color variation results are sent to the terminal and notified to the user.

[0630] 7. The device displays the results and receives feedback

[0631] User A checks the proposed styles, selects the one they like, and enters their feedback, which is then sent to the server.

[0632] Prompt Sentence Examples

[0633] "Based on the photo below, create hairstyle suggestions and color variations for a short bob. The hair texture is straight and the hair volume is medium."

[0634] This system allows users to receive suggestions for hairstyles that best suit their preferences, and also allows them to further customize their style based on a wide range of color variations and trend information.

[0635] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0636] Step 1: User Login and Authentication

[0637] When the user launches the app, the device displays the login screen. The user enters their username and password and presses the "Login" button. The server receives the username and password sent from the device and authenticates them by checking them against the database. The input data are the username and password, and the output is the success or failure of the authentication.

[0638] Step 2: Upload photos and enter your request

[0639] If authentication is successful, the device displays an upload screen. The user takes a photo of themselves or selects one from the gallery and uploads it. After the photo is uploaded, the device displays a request input screen. The user enters desired information such as hair volume, texture, length, and style. The device sends the uploaded photo and request data to the server. The input is the photo data and request data, and the output is the data sent to the server.

[0640] Step 3: Data analysis

[0641] The server stores the photo data and request data received from the device in a database. Next, the server analyzes the photo data using image recognition technology (e.g., TensorFlow) to extract the user's hair characteristics. For example, it analyzes information such as hair length, volume, texture, and color. The input is the photo data and request data, and the output is hair feature data.

[0642] Step 4: Hairstyle generation

[0643] The server inputs data into a generative AI model (e.g., GAN) based on the analyzed hair characteristics and the user's request. The model generates the optimal hairstyle and stores the results in an internal database. The input is hair characteristic data and request data, and the output is the generated hairstyle data.

[0644] Step 5: Collect trend information and make additional suggestions

[0645] The server accesses an external API (e.g., FashionTrendAPI) to collect the latest hairstyle trend information. It analyzes the collected trend information and reflects it in the generated hairstyle to make additional suggestions. The input is trend information, and the output is hairstyle data that reflects the trend.

[0646] Step 6: Coloring simulation

[0647] The server inputs data into a color model (e.g., OpenCV) for the generated hairstyle. The model generates multiple color variations and stores them in an internal database. The input is the generated hairstyle data, and the output is the color variation data.

[0648] Step 7: Submit your proposal

[0649] The server compiles the created hairstyle and color variation results and sends the proposal results in JSON format to the device. The device receives this and displays the proposal to the user. The input is hairstyle data and color variation data, and the output is the proposal result data.

[0650] Step 8: Viewing and Feedback on Proposal Results

[0651] The user checks the proposed hairstyles and color variations on the device, selects the style they like, and enters their feedback. The device then sends this feedback to the server. The input is the user's selection and feedback, and the output is the feedback data.

[0652] (Application example 1)

[0653] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0654] Conventional hair style suggestion systems for beauty salons are capable of generating hairstyles based on image analysis and customer requests, but it is difficult to propose these results to customers in real time. Furthermore, they lack support for customers to visually visualize the proposed style. Therefore, there is a need for a method that combines real-time performance and visual clarity, both of which are necessary for proposing hairstyles that satisfy customers.

[0655] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0656] In this invention, the server includes means for receiving image data of a customer, means for receiving a hairstyle request from the customer, means for analyzing the image data to extract characteristics of the customer's hair, means for generating an optimal hairstyle based on the customer's requests and hair characteristics, means for proposing the generated hairstyle to the customer, and means for displaying the generated hairstyle in real time using a wearable device such as smart glasses, thereby enabling visually easy-to-understand hairstyle suggestions and virtual coloring simulations to be provided to the customer in real time.

[0657] The "means for receiving image data of a customer" is an interface for inputting or uploading image data including a customer's facial photograph and hair characteristics in the beauty salon system.

[0658] The "means for receiving customer requests regarding hairstyle" is an interface for the customer to input or select styling requirements such as the desired hairstyle and coloring.

[0659] "Means for analyzing image data and extracting characteristics of a customer's hair" refers to algorithms or technologies for analyzing received image data of a customer and identifying characteristics such as hair length, volume, texture, and color.

[0660] The "means for generating the optimal hairstyle based on the customer's requests and hair characteristics" is an algorithm for designing and generating the optimal hairstyle using an AI model based on the extracted hair characteristics and customer requests.

[0661] The "means for proposing the generated hairstyle to the customer" is a display interface for visually presenting the hairstyle generated by the system to the customer and obtaining selection and feedback.

[0662] "Means for displaying the generated hairstyle in real time using a wearable device such as smart glasses" refers to technology that uses a wearable device such as smart glasses worn by a hairdresser to show the generated hairstyle to the customer in real time.

[0663] The "means for performing virtual coloring simulation" is a technology for applying multiple color variations to a generated hairstyle and visually simulating the results.

[0664] "Means of collecting the latest trend information and proposing additional hairstyles to customers based on the trends" is a function that collects the latest hairstyle trend information from external APIs and databases and makes additional suggestions to customers based on those trends.

[0665] The present invention provides a specific method for building a system for a beauty salon to suggest customized hairstyles to customers, which can display and suggest optimal hairstyles to customers in real time using a wearable device such as smart glasses.

[0666] 1. Hardware and Software Configuration

[0667] The system consists of the following major hardware and software components:

[0668] Hardware:

[0669] Smart glasses (e.g., Google Glass, Vuzix Blade)

[0670] Cloud Server

[0671] software:

[0672] Cloud-based data processing systems (e.g., AWS Lambda, Microsoft Azure)

[0673] Image recognition technology (e.g., OpenCV, Google Cloud Vision API)

[0674] Hairstyle generation system using AI models (e.g. TensorFlow, PyTorch)

[0675] Voice recognition technology (e.g., Google Speech-to-Text API)

[0676] 2. System Operation Overview

[0677] The system works as follows:

[0678] 1. Collection of customer image data

[0679] A hairdresser puts on smart glasses and uses the built-in camera to take a photo of the customer's face.

[0680] After the photo is taken, the image data is sent to a cloud server.

[0681] 2. Gathering customer requirements

[0682] The hairdresser uses voice recognition to input the customer's desired hairstyle and coloring requests.

[0683] The request data is also sent to the cloud server.

[0684] 3. Analysis of image data

[0685] The cloud server analyzes the received image data and extracts the customer's hair characteristics (length, volume, texture, color, etc.).

[0686] Image recognition technologies used include OpenCV and Google Cloud Vision API.

[0687] 4. Hairstyle generation

[0688] Based on the analyzed hair characteristics and the customer's requests, an AI model is used to generate the optimal hairstyle.

[0689] The AI ​​models used include TensorFlow and PyTorch.

[0690] 5. Virtual Coloring Simulation

[0691] A virtual coloring simulation is performed by applying multiple color variations to the generated hairstyle.

[0692] The simulation results are also stored on the cloud server.

[0693] 6. Real-time display of results

[0694] The generated hairstyle and color variation results are sent to the smart glasses.

[0695] Hairdressers make real-time suggestions to customers through smart glasses.

[0696] Specific examples

[0697] Example 1: Proposing a hairstyle for customer A

[0698] 1. The hairdresser puts on the smart glasses and takes a photo of Customer A's face.

[0699] 2. Using the voice recognition function, Customer A inputs the style he or she desires (e.g., short bob, brown color).

[0700] 3. The captured image data and requested data are sent to the cloud server.

[0701] 4. The image data is analyzed on the cloud server, and hair characteristics (e.g., medium length, straight, black hair) are extracted.

[0702] 5. Based on the analysis results and your request, the AI ​​model will generate three short bob styles and perform a virtual coloring simulation.

[0703] 6. The results are sent to the smart glasses, and the hairdresser makes suggestions to Customer A through the glasses.

[0704] 7. Customer A chooses the best style and enters feedback via voice recognition.

[0705] Prompt Sentence Examples

[0706] To ask an AI model to generate a hairstyle based on a customer's request, use the following prompt:

[0707] "Generate a suitable hairstyle based on the customer's facial photo and requests."

[0708] This process allows customers to visually see in real time which hairstyle best suits them, ensuring a highly satisfying service.

[0709] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0710] Step 1:

[0711] Input: A customer comes in and the hairdresser puts on the smart glasses.

[0712] What it does: A hairdresser uses the built-in camera in the smart glasses to take a photo of the customer's face.

[0713] Output: The captured image data is stored in the smart glasses.

[0714] Step 2:

[0715] Input: A photo of the customer's face.

[0716] Specific operation: A photo of the customer's face is sent from the smart glasses to a cloud server.

[0717] Output: The cloud server receives and stores the facial photo data.

[0718] Step 3:

[0719] Input: Customer's hairstyle and coloring requests.

[0720] Specific operation: The hairdresser uses the voice recognition function to input the customer's request by voice.

[0721] Output: The audio data is sent to a cloud server and converted into text data.

[0722] Step 4:

[0723] Input: Customer's photo data and request data.

[0724] How it works: The cloud server uses image recognition technology (e.g., OpenCV or Google Cloud Vision API) to analyze the customer's hair characteristics (length, volume, texture, color, etc.).

[0725] Output: Analysis data containing hair features is generated and saved.

[0726] Step 5:

[0727] Input: Customer's hair characteristics analysis data and request data.

[0728] What it does: A cloud server uses an AI model (e.g., TensorFlow or PyTorch) to generate the optimal hairstyle.

[0729] Output: The generated hairstyle data is saved.

[0730] Step 6:

[0731] Input: Generated hairstyle data.

[0732] Specific operation: The cloud server performs a virtual coloring simulation and applies multiple color variations.

[0733] Output: Hairstyle data including coloring simulation results is generated and saved.

[0734] Step 7:

[0735] Input: Hairstyle data including coloring simulation results.

[0736] Specific operation: The cloud server sends hairstyle data to the smart glasses.

[0737] Output: The transmitted hairstyle data is displayed on the smart glasses.

[0738] Step 8:

[0739] Input: Hairstyle data displayed on smart glasses.

[0740] How it works: A hairdresser uses smart glasses to suggest the best hairstyle and color options to a customer in real time.

[0741] Output: The customer reviews the suggested hairstyles and selects the best one.

[0742] Step 9:

[0743] Input: Customer selection and feedback.

[0744] Specific operation: The hairdresser inputs the customer's selection and feedback through voice recognition in the smart glasses.

[0745] Output: The feedback data is sent to the cloud server and stored.

[0746] This allows the server and terminal to work together, making it possible to suggest the best hairstyle for each customer in real time.

[0747] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0748] This invention is an application system for beauty salons that proposes hairstyles customized to meet customer needs, and provides more personalized services by combining it with an emotion engine that recognizes the user's emotions. This system analyzes the customer's image data, generates hairstyles according to their requests, and makes suggestions based on coloring simulations and trend information. The emotion engine also recognizes the user's emotions and optimizes the suggestions based on these.

[0749] Overall system configuration

[0750] The system consists of the following main components:

[0751] 1. Photo upload system (terminal): A user interface for customers to upload their own images.

[0752] 2. Request input system (terminal): An interface for customers to input desired styling elements and specific requests.

[0753] 3. Data analysis system (server): Technology for analyzing received image data and extracting the characteristics of the customer's hair.

[0754] 4. Generation system (server): An algorithm for generating the optimal hairstyle based on the extracted hair characteristics and customer requests.

[0755] 5. Proposal system (terminal): An interface for proposing the generated hairstyle to the customer.

[0756] 6. Coloring Simulation System (Server): Technology that simulates virtual coloring.

[0757] 7. Trend information collection and suggestion system (server): A function that collects the latest trend information and suggests additional hairstyles based on it.

[0758] 8. Emotion Engine (Server): Technology that recognizes user emotions and adjusts suggestions based on them.

[0759] Program Processing Overview

[0760] 1. Upload photos and input requests (device processing)

[0761] After launching the application and logging in, the user is presented with a photo upload screen. The user can select a photo from their photo library or take a new photo and upload it. A styling request input screen is then displayed, where the user enters information such as the desired hair volume, texture, length, and style, and the data is sent to the server.

[0762] 2. Data analysis (server processing)

[0763] The server stores the photo data and requests received from the device in a database, and then uses image recognition technology to extract the customer's hair characteristics, such as hair length, volume, texture, and color.

[0764] 3. Hairstyle generation (server processing)

[0765] The server uses an AI model to generate the optimal hairstyle based on the analyzed hair characteristics and the customer's requests, and the generated style is stored in an internal database.

[0766] 4. Collecting trend information and making additional suggestions (server processing)

[0767] The server uses external APIs and internal databases to collect the latest hairstyle trend information and provide additional hairstyle suggestions based on that information, which are also stored in the database.

[0768] 5. Coloring simulation (server processing)

[0769] The server performs a virtual coloring simulation on the generated hairstyle, using an AI model to generate and store multiple color variations that match different hair colors and skin tones.

[0770] 6. Emotion Recognition and Suggestion Adjustment (Server Processing)

[0771] The server uses an emotion engine to analyze the facial expressions and voice of the user reviewing the proposed hairstyle and recognize their emotions. Based on the recognized emotions, the server adjusts the proposal and provides personalized suggestions.

[0772] 7. Sending the proposal results (server processing)

[0773] The server compiles the generated hairstyle suggestions, coloring simulation results, and suggestions adjusted by emotion recognition, and sends them to the terminal, which receives the suggestions and displays them to the customer.

[0774] 8. Displaying and Feedback of Proposal Results (Device Processing)

[0775] The user can check the proposed hairstyles and color variations on the device, select the style they like, and enter their feedback, which is then sent to the server.

[0776] 9. Saving feedback and reflecting it in the next proposal (server processing)

[0777] The server stores the received feedback in a database and uses it for future suggestions. This feedback is also used as training data for the AI ​​model, and is used to increase user satisfaction.

[0778] Specific examples

[0779] Example 1: Proposing a new hairstyle for Customer B

[0780] 1. User uploads a photo

[0781] Customer B launches the application on the device, uploads a photo of himself, and inputs his styling requests: hair volume: thick, texture: wavy, desired length: medium, style preference: casual.

[0782] 2. The server analyzes the data

[0783] Using image recognition technology, the characteristics of Customer B's hair are extracted. The analysis reveals that the hair volume is thick, the length is medium, and the color is black.

[0784] 3. The server generates the hairstyle

[0785] Generate three casual styles based on the customer's requests and hair characteristics.

[0786] 4. The server acquires trend information

[0787] We collect the latest trend information and reflect the latest trends in casual style.

[0788] 5. The server runs the coloring simulation.

[0789] Generates four different color variations (e.g., natural black, dark brown, light brown, reddish brown).

[0790] 6. Emotion recognition and suggestion adjustment

[0791] As the user reviews the suggested styles, an emotion engine analyzes the user's facial expressions. For example, if surprise or joy is detected, the server will adjust the suggestions based on that emotion and make other suggestions as well.

[0792] 7. The server sends the proposal results

[0793] Suggested hairstyles and color variations, tailored suggestions based on emotions, are sent to the device.

[0794] 8. The device displays the results and receives feedback

[0795] Customer B reviews the proposed styles, selects the style they like, and provides feedback.

[0796] 9. The server stores the feedback

[0797] Receive feedback and incorporate it into your next proposal.

[0798] This allows Customer B to not only receive suggestions for the optimal hairstyle and coloring simulation that suits his or her preferences, but also receive more personalized suggestions based on his or her emotions, resulting in high levels of satisfaction.

[0799] The processing flow will be explained below.

[0800] Step 1:

[0801] When a user launches the application on their device, the login screen appears. The user enters their email address and password and taps the "Login" button.

[0802] Step 2:

[0803] The terminal sends login information to the server. The server checks the received login information against a database, and if authentication is successful, starts a session and responds to the terminal. If authentication fails, it returns an error message.

[0804] Step 3:

[0805] After successful login, the device will display the main screen and present the user with a photo upload screen where they can select a photo from their photo library or take a new photo and upload it.

[0806] Step 4:

[0807] Once the user selects or takes a photo, the device will then display a styling request input screen, where the user can enter the desired hair volume, texture, length, styling preferences, etc., and tap the "Send" button.

[0808] Step 5:

[0809] The device sends the uploaded photos and entered styling requests to the server, which receives this data and stores it in a database.

[0810] Step 6:

[0811] The server uses image recognition technology to analyze the received photo data and extract the customer's hair characteristics (length, volume, texture, color, etc.), which are then stored in a database.

[0812] Step 7:

[0813] The server uses an AI model to generate the optimal hairstyle based on the customer's request and the results of image analysis. The AI ​​model also takes into account past data and trend information when proposing a style.

[0814] Step 8:

[0815] The server uses external APIs and internal databases to collect the latest hairstyle trend information and makes additional hairstyle suggestions based on that information, which are also stored in the database.

[0816] Step 9:

[0817] The server then runs a virtual coloring simulation on the generated hairstyle, using an AI model to generate multiple color variations that match the hair color and skin tone, and stores the results.

[0818] Step 10:

[0819] The server uses an emotion engine to analyze the facial expressions and voice of the user confirming the proposed hairstyle and recognize their emotions. For example, emotions such as joy or dissatisfaction can be extracted from the user's facial expressions and voice.

[0820] Step 11:

[0821] The server then tailors the suggestions based on the recognized emotion, for example, by prioritizing hairstyles that indicate a happy emotion to the user.

[0822] Step 12:

[0823] The server sends the generated hairstyle suggestions, coloring simulation results, and suggestions adjusted by emotion recognition to the terminal, which receives the suggestions and displays them to the customer.

[0824] Step 13:

[0825] The user can check the proposed hairstyles and color variations on the device, select the style they like, and enter their feedback, which is then sent to the server.

[0826] Step 14:

[0827] The server stores the received feedback in a database and uses it for future suggestions. This feedback is also used as training data for the AI ​​model.

[0828] In this way, the entire system works together to suggest the best hairstyle for the user and improve the quality of service based on emotion recognition and feedback.

[0829] Example 2

[0830] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0831] While existing hair salon application systems suggest hairstyles based on customer requests, they lack personalized suggestions that take into account customer emotions and trend information. Furthermore, there is no system in place to incorporate feedback on suggested styles into future suggestions, which means customer satisfaction is not fully improved. Furthermore, coloring simulations are limited, making it difficult to fully address the diverse needs of customers.

[0832] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0833] In this invention, the server includes a means for receiving image data of a customer, a means for receiving the customer's hairstyle requests, and a means for analyzing the image data to extract the customer's hair characteristics. This makes it possible to generate and propose an optimal hairstyle based on the customer's requests and hair characteristics. The server also includes a means for recognizing the user's emotions and optimizing the proposal based on the emotions, a means for receiving feedback from the customer and reflecting it in future proposals, a means for performing a virtual coloring simulation for the generated hairstyle, and a means for collecting the latest trend information and proposing additional hairstyles based on the trends to the customer. This makes it possible to propose hairstyles that are personalized to the customer, thereby increasing customer satisfaction.

[0834] The "means for receiving image data from customers" is a function for acquiring images uploaded by customers, converting them into a format that can be used within the system, and saving them.

[0835] The "means for receiving customer requests regarding hairstyle" is a function for inputting information regarding the hairstyle desired by the customer, and collecting and saving the information.

[0836] The "means for analyzing image data and extracting characteristics of the customer's hair" is a function for identifying and extracting characteristics such as hair length, volume, texture, and color based on the received image data.

[0837] "A means for generating the optimal hairstyle based on the customer's requests and hair characteristics" is a function that uses an AI model to generate multiple candidate hairstyles based on the customer's requests and the results of data analysis.

[0838] The "means for proposing generated hairstyles to a customer" is a function for presenting generated hairstyles to a customer and encouraging them to select and evaluate them.

[0839] "Means for recognizing the user's emotions and optimizing suggestions based on them" refers to a function that analyzes the user's facial expressions and voice data and adjusts the suggestions based on their emotional state.

[0840] "Means of receiving feedback from customers and reflecting it in future proposals" is a function that records the evaluations and comments that customers make on proposals and uses them to help with future proposals.

[0841] The "means for performing a virtual coloring simulation for the generated hairstyle" is a function for simulating different hair colors for the generated hairstyle and presenting them to the customer.

[0842] "Means for collecting the latest trend information and proposing additional hairstyles based on the trends to customers" refers to a function for obtaining the latest hairstyle trend data from an external source and proposing additional hairstyles based on that data to customers.

[0843] This invention is an application system for beauty salons that proposes hairstyles customized to the customer's needs and provides more personalized services by combining it with an emotion engine that recognizes the user's emotions. The system consists of the following main components:

[0844] 1. Photo upload system (terminal)

[0845] This is a user interface for customers to upload their own images. Users launch the app on their smartphone, log in, and then select a photo from their photo library or take a new photo and upload it.

[0846] 2. Request input system (terminal)

[0847] This is the interface where the customer inputs the styling elements and specific requests they want, such as hair volume, texture, length, and style, and then sends the data to the server.

[0848] 3. Data analysis system (server)

[0849] This technology analyzes the received image data and extracts the customer's hair characteristics. This process uses the image recognition library OpenCV and the deep learning framework TensorFlow. For example, features such as hair length, volume, texture, and color can be extracted from the image.

[0850] 4. Generation System (Server)

[0851] Based on the analyzed hair characteristics and the customer's requests, an optimal hairstyle is generated using an AI model (e.g., a generative artificial network (GAN)). The generated style is saved in an internal database. An example of a prompt sentence is, "The client has a lot of hair, the texture is wavy, the desired length is medium, and the style preference is casual."

[0852] 5. Proposed system (terminal)

[0853] This is an interface for proposing generated hairstyles to customers. Users can check multiple hairstyles generated on the app and enter feedback. The feedback is sent to the server.

[0854] 6. Coloring Simulation System (Server)

[0855] A virtual coloring simulation is performed on the generated hairstyle, using an AI model to generate multiple color variations (e.g., natural black, dark brown, light brown, reddish brown) that match the hair color and skin tone, and then stored in a database.

[0856] 7. Trend information collection and proposal system (server)

[0857] This function collects the latest trend information from external APIs (e.g., SNS trend APIs) and internal databases, and then suggests additional hairstyles based on that information. Trend information is updated regularly, and new styles are suggested to customers.

[0858] 8. Emotion Engine (Server)

[0859] This technology recognizes the user's emotions and optimizes suggestions based on them. It uses the Emotion API to analyze the user's facial expressions and voice to recognize their emotional state. For example, if surprise or joy is detected, the system reevaluates the suggestions and makes personalized suggestions.

[0860] 9. Feedback Collection System (Server)

[0861] This function receives customer feedback on the proposed style and reflects it in future proposals. The received feedback is stored in a database and used as learning data for the AI ​​model.

[0862] This allows the system to not only propose optimal hairstyles based on the customer's requests and hair characteristics, but also provide personalized services that take into account the user's emotions and trend information.Furthermore, by incorporating feedback into the next proposal, customer satisfaction can be further increased.

[0863] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0864] Step 1:

[0865] User

[0866] The user launches the application and logs in. After logging in, a photo upload screen is displayed. The user selects a photo from the photo library or takes a new photo using the device's camera. The input is the user's photo data, and the output is the uploaded image data. Specifically, the user taps the smartphone app to launch it, and then enters their username and password for authentication. After authentication, the user selects or takes a photo and sends it to the server.

[0867] Step 2:

[0868] server

[0869] The server receives the uploaded image data and stores it in a database. It also analyzes it for the next step. The input is the uploaded image data, and the output is the image data stored in the database. Specifically, it receives an HTTP request and stores the image data in the database.

[0870] Step 3:

[0871] User

[0872] The user moves to the styling request input screen, enters information such as hair volume, texture, length, and style, and sends it to the server. The input is the request data entered by the user, and the output is the request data sent. Specifically, the user selects hair characteristics on the request input screen and taps the "Send" button.

[0873] Step 4:

[0874] server

[0875] The server stores the received request data in a database and analyzes it together with the image data. The input is the image data and the request data, and the output is the analyzed hair feature data. This analysis is performed using OpenCV and TensorFlow. Specifically, an image recognition algorithm is used to identify features such as hair length, volume, texture, and color.

[0876] Step 5:

[0877] server

[0878] The server uses an AI model (e.g., GAN) to generate the optimal hairstyle based on the analyzed hair features and desired data. The input is the analyzed hair feature data and desired data, and the output is the generated hairstyle data. Specifically, the server inputs a prompt statement into the AI ​​model to generate multiple hairstyles.

[0879] Step 6:

[0880] server

[0881] The server saves the generated hairstyle data in a database and sends it to the device. The input is the generated hairstyle data, and the output is the hairstyle suggestion data sent to the device. The specific operation is to save the results in a database and send them to the device as an HTTP response.

[0882] Step 7:

[0883] User

[0884] The user reviews the hairstyle suggestions generated on the device and enters feedback. The input is the proposed hairstyle data, and the output is the feedback data. Specifically, the user browses the suggested hairstyles on the app, selects them, and enters comments.

[0885] Step 8:

[0886] server

[0887] The server stores the received feedback data in a database and reflects it in the next proposal. The input is the feedback data, and the output is an updated database. Specifically, the server analyzes the feedback information and updates the database to use it in the next proposal.

[0888] Step 9:

[0889] server

[0890] The server uses an external API to collect the latest trend information. The input is the trend information obtained from the external API, and the output is the trend information stored in the database. Specifically, the server periodically sends an API request to obtain and store the latest trend information data.

[0891] Step 10:

[0892] server

[0893] The server performs a virtual coloring simulation for the generated hairstyle. The input is the generated hairstyle data and the user's skin tone data, and the output is multiple color variation data. Specifically, the server uses the coloring simulation model to generate different color variations of the hairstyle and save them in a database.

[0894] Step 11:

[0895] server

[0896] The server analyzes the user's facial expressions and voice to recognize emotions. The input is the user's facial expression and voice data, and the output is the recognized emotion data. Specifically, it calls the Emotion API, analyzes the user's facial expression and voice data, and identifies the emotion.

[0897] Step 12:

[0898] server

[0899] The server optimizes the suggestions based on the recognized emotion. The input is the recognized emotion data and the suggestion data, and the output is the suggestion data adjusted based on the emotion. Specifically, the server uses the AI ​​model to reevaluate the suggestions and reconfigure the optimal hairstyle based on the emotion.

[0900] (Application example 2)

[0901] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0902] Conventional hair salon application systems suggest hairstyles based on customer requests, but do not provide personalized suggestions based on the customer's emotions, making it difficult to maximize customer satisfaction. In addition to simply suggesting hairstyles, salons are also required to provide coloring simulations and trend information, but there is a lack of systems that provide these functions in an integrated manner. Furthermore, there is a need for a system that can incorporate real-time emotion recognition to make optimal suggestions based on the customer's emotions.

[0903] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0904] In this invention, the server includes a means for receiving image data of a customer, a means for receiving the customer's hairstyle requests, a means for analyzing the image data to extract the customer's hair characteristics, a means for generating an optimal hairstyle based on the customer's requests and hair characteristics, a means for proposing the generated hairstyle to the customer, and a means for analyzing the customer's facial expressions and voice to recognize emotions and adjust the proposed result. This makes it possible to propose an optimal hairstyle based on the customer's emotions in real time, in addition to their requests. It is also possible to perform coloring simulations and propose additional hairstyles based on the latest trend information, which can significantly improve overall customer satisfaction.

[0905] The "means for receiving customer image data" is a device that provides an interface for users to upload their photos to the system.

[0906] The "means for receiving customer requests regarding hairstyle" is a device that provides an interface through which the user can input information regarding the hairstyle and coloring they desire.

[0907] The "means for analyzing image data to extract characteristics of the customer's hair" is a device that includes an algorithm that analyzes uploaded image data and identifies characteristics such as hair length, texture, and color.

[0908] The "means for generating the optimal hairstyle based on the customer's requests and hair characteristics" is a device that integrates the user's requests with analyzed hair characteristics and generates the optimal hairstyle using an AI model.

[0909] The "means for proposing generated hairstyles to a customer" is a device that presents the hairstyles generated by the system to the user and provides an interface for receiving selections and feedback.

[0910] The "means for analyzing the customer's facial expressions and voice to recognize emotions and adjust the suggested results" refers to a device that includes an algorithm that analyzes the user's facial expressions and voice, recognizes emotions in real time, and adjusts the suggested hairstyle based on the results.

[0911] This invention relates to a system for improving customer experience in beauty salons. Specifically, the system receives customer image data and requests, analyzes them to suggest hairstyles and colorings, and recognizes emotions to adjust the suggestions, providing a more personalized service.

[0912] The system includes, among other things, the following main components:

[0913] 1. Means of receiving customer image data

[0914] It is mainly implemented in devices such as smartphones and tablets, and provides an interface for users to upload their own photos. For example, a smartphone application allows a user to take a selfie with their camera and upload it to the system. This photo data is then sent to a cloud server.

[0915] 2. A means of receiving customer requests regarding hairstyles

[0916] It provides an interface for users to input their desired hairstyle and coloring information. For example, users can input detailed requests such as hair length, texture, color, and style preferences into the app.

[0917] 3. A method for analyzing image data to extract the characteristics of a customer's hair

[0918] Using image recognition technology, the uploaded photo is analyzed to extract features such as hair length, texture, and color. This analysis can be performed using libraries such as TensorFlow and OpenCV. The analysis results are used in the next step.

[0919] 4. A means to generate optimal hairstyles based on customer requests and hair characteristics

[0920] Using a generative AI model, we generate the optimal hairstyle based on the user's request and analyzed hair characteristics. We use deep learning frameworks such as Keras and PyTorch to propose several hairstyle candidates.

[0921] 5. A means of proposing generated hairstyles to customers

[0922] The proposed hairstyles are presented to the user and an interface is provided for selection and feedback. For example, the generated hairstyles are displayed on the app, and the user can browse them and select the style they prefer.

[0923] 6. A method to recognize emotions by analyzing customer facial expressions and voice and adjust the proposal results

[0924] As users review the suggested hairstyles, the system analyzes their facial expressions and voice to recognize their emotions. This process is carried out using emotion recognition software such as EmotionEngine. Based on the recognized emotions, the system adjusts the suggestions to improve user satisfaction.

[0925] Specific examples

[0926] Let's take a specific example where customer C is looking for a new hairstyle at a hair salon.

[0927] 1. Upload a photo

[0928] Customer C opens the app, takes a photo of themselves, and uploads it.

[0929] 2. Input your request

[0930] Hair length: Medium

[0931] Hair Texture:Wave

[0932] Favorite color: Dark

[0933] 3. Data Analysis

[0934] It is analyzed as follows: Length: Medium, Texture: Wavy, Color: Black.

[0935] 4. Hairstyle Generation

[0936] Based on the analysis results, several new styles are generated.

[0937] 5. Emotion recognition

[0938] Recognize the happy expression on Customer C's face when he sees the proposed style.

[0939] 6. Final proposal result

[0940] The tailored suggestions are displayed to Customer C on her smartphone, and she chooses the style she likes best.

[0941] Prompt Sentence Examples

[0942] Image file: "path_to_your_image_file.jpg"

[0943] Customer Request:

[0944] Hair Length: Medium

[0945] Hair Texture: Wavy

[0946] Favorite color: Dark

[0947] Suggested style: "Curl"

[0948] Emotion recognition result: "Joy"

[0949] Make your final offer based on emotion.

[0950] This allows users to not only receive suggestions for the best hairstyle and coloring simulation based on their needs, but also receive more personalized suggestions based on their emotions, providing a highly satisfying experience.

[0951] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0952] Step 1:

[0953] Upload a photo

[0954] Subject: User

[0955] Description: A user launches the application, takes a photo of themselves, and uploads it to the system. The input is image data taken with the smartphone camera, and the output is data in an image file format sent to the cloud server. Specifically, the user presses the "Upload Photo" button, selects a photo, and uploads it.

[0956] Step 2:

[0957] Request input

[0958] Subject: User

[0959] Description: A user uses an application interface to input detailed information about their desired hairstyle and color. The input is the user's style preferences provided through text and options, and the output is data about those preferences sent to the server. Specific operations include selecting length, texture, color, etc. on the "Enter Hairstyle Preferences" screen and submitting it.

[0960] Step 3:

[0961] Data analysis

[0962] Subject: Server

[0963] Description: The server analyzes and processes the image data received and extracts the user's hair features. The input is the uploaded image data, and the output is the analysis results such as hair length, texture, and color. Specifically, it uses TensorFlow and OpenCV to detect hair areas from the image and extracts their features as numerical data.

[0964] Step 4:

[0965] Hairstyle generation

[0966] Subject: Server

[0967] Description: The server uses a generative AI model to generate the optimal hairstyle based on the analyzed hair features and the user's requests. The input is hair feature data and user request data, and the output is generated hairstyle candidates. Specifically, it applies a trained model using Keras or PyTorch to generate several optimal hairstyles.

[0968] Step 5:

[0969] Providing proposal results

[0970] Subject: Server

[0971] Description: The generated hairstyle is proposed to the user and made available for viewing. The input is the data of the generated hairstyle, and the output is the proposal information sent to the user's device. Specifically, the application generates images of the multiple generated hairstyles and displays them on the application screen.

[0972] Step 6:

[0973] emotion recognition

[0974] Subject: Server

[0975] Description: The server analyzes the user's facial expressions and voice to recognize emotions. The input is the user's facial expression and voice data, and the output is the recognized emotion data. Specifically, it uses the Emotion Engine to analyze data obtained from the camera and microphone and quantifies the user's emotions.

[0976] Step 7:

[0977] Adjustment of proposed results

[0978] Subject: Server

[0979] Description: Adjusts suggested hairstyles based on emotion recognition results. The input is the recognized emotion data and initial hairstyle suggestion data, and the output is the adjusted final suggestion data. Specific operations include modifying suggested hairstyles or making additional suggestions based on the recognized emotion.

[0980] Step 8:

[0981] Provision of final proposal results

[0982] Subject: Server

[0983] Description: Provides the user with a final hairstyle proposal after adjustments. The input is the adjusted proposal data, and the output is the final proposal information sent to the user's device. Specific operations include displaying the adjusted hairstyle proposal in the application so that the user can confirm it.

[0984] Step 9:

[0985] Collecting and storing feedback

[0986] Subject: User and Server

[0987] Description: The user reviews the suggested styles, selects the one they like, and provides feedback. This feedback is sent to the server and saved. The input is the hairstyle selected by the user and the feedback data, and the output is the feedback data saved on the server. Specifically, when the user presses the "Send Feedback" button, the information entered is sent to the server and used for the next suggestion.

[0988] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0989] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0990] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[0991] [Third embodiment]

[0992] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[0993] 5, the data processing system 310 includes the data processing device 12 and a headset type terminal 314. An example of the data processing device 12 is a server.

[0994] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0995] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[0996] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0997] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0998] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0999] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1000] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1001] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1002] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1003] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[1004] This invention is a system for building an application for beauty salons that proposes customized hairstyles that meet customer needs. This system analyzes the customer's image data, generates the optimal hairstyle, and provides coloring simulations and the latest trend information.

[1005] Overall system configuration

[1006] The system consists of the following main components:

[1007] 1. Photo upload system (terminal): A user interface for customers to upload their own images.

[1008] 2. Request input system (terminal): An interface for customers to input desired styling elements and specific requests.

[1009] 3. Data analysis system (server): Technology for analyzing received image data and extracting the characteristics of the customer's hair.

[1010] 4. Generation system (server): An algorithm for generating the optimal hairstyle based on the extracted hair characteristics and customer requests.

[1011] 5. Proposal system (terminal): An interface for proposing the generated hairstyle to the customer.

[1012] 6. Coloring Simulation System (Server): Technology that simulates virtual coloring.

[1013] 7. Trend information collection and suggestion system (server): A function that collects the latest trend information and suggests additional hairstyles based on it.

[1014] Program Processing Overview

[1015] 1. Upload photos and input requests (device processing)

[1016] The customer launches the application and logs in. After logging in, a photo upload screen appears, where the customer can take or select a photo of themselves. Next, a screen appears where the customer can enter information such as the amount, texture, length, and style of their desired hair. This data is then sent from the device to the server.

[1017] 2. Data analysis (server processing)

[1018] The server stores the photo data and requests received from the device in a database, and then uses image recognition technology to extract the customer's hair characteristics, such as hair length, volume, texture, and color.

[1019] 3. Hairstyle generation (server processing)

[1020] The server uses an AI model to generate the optimal hairstyle based on the analyzed hair characteristics and the customer's requests, and the generated style is stored in an internal database.

[1021] 4. Collecting trend information and making additional suggestions (server processing)

[1022] The server uses external APIs and internal databases to collect the latest hairstyle trends, and then provides additional hairstyle suggestions based on these trends.

[1023] 5. Coloring simulation (server processing)

[1024] The server then performs a virtual coloring simulation on the generated hairstyle, using an AI model to generate and save color variations that match the hair color and skin tone.

[1025] 6. Sending the proposal results (server processing)

[1026] The server sends the hairstyle proposal results and coloring simulation results to the terminal, which receives them and displays them to the customer.

[1027] 7. Displaying and Feedback of Proposal Results (Device Processing)

[1028] The customer checks the proposed hairstyles and color variations on the terminal and inputs their selection or feedback, which is then sent to the server.

[1029] Specific examples

[1030] Example 1: Proposing a new hairstyle to Customer A

[1031] 1. User uploads a photo

[1032] Customer A launches the application on the device, uploads a photo of himself, and inputs his hair volume: normal, texture: straight, desired length: short, and style preference: bob.

[1033] 2. The server analyzes the data

[1034] Using image recognition technology, the characteristics of Customer A's hair are extracted. The analysis reveals that hair volume is normal, length is medium, and color is brown.

[1035] 3. The server generates the hairstyle

[1036] Generate three bob styles based on the client's requests and hair characteristics.

[1037] 4. The server acquires trend information

[1038] We collect the latest trend information and reflect the trends in bob styles from that information.

[1039] 5. The server runs the coloring simulation.

[1040] Generates four color variations (e.g., natural brown, light brown, dark brown, reddish brown).

[1041] 6. The server sends the proposal results

[1042] Suggested hairstyles and color variations are sent to your device.

[1043] 7. The device displays the results and receives feedback

[1044] Customer A reviews the proposed styles, selects the style they like, and enters their feedback.

[1045] This allows customer A to receive suggestions for the best hairstyle that suits his or her desires.

[1046] The processing flow will be explained below.

[1047] Step 1:

[1048] When a user launches the application on their device, the login screen appears. The user enters their email address and password and taps the "Login" button.

[1049] Step 2:

[1050] The terminal sends login information to the server. The server checks the received login information against a database, and if authentication is successful, starts a session and responds to the terminal. If authentication fails, it returns an error message.

[1051] Step 3:

[1052] After successful login, the device will display the main screen and present the user with a photo upload screen where they can select a photo from their photo library or take a new photo and upload it.

[1053] Step 4:

[1054] Once the user selects or takes a photo, the device will then display a styling request input screen, where the user can enter the desired hair volume, texture, length, styling preferences, etc., and tap the "Send" button.

[1055] Step 5:

[1056] The device sends the uploaded photos and entered styling requests to the server, which receives this data and stores it in a database.

[1057] Step 6:

[1058] The server uses image recognition technology to analyze the received photo data and extract the customer's hair characteristics (length, volume, texture, color, etc.), which are then stored in a database.

[1059] Step 7:

[1060] The server uses an AI model to generate the optimal hairstyle based on the customer's request and the results of image analysis. The AI ​​model also takes into account past data and trend information when proposing a style.

[1061] Step 8:

[1062] The server uses external APIs and internal databases to collect the latest hairstyle trend information and makes additional hairstyle suggestions based on that information, which are also stored in the database.

[1063] Step 9:

[1064] The server then runs a virtual coloring simulation on the generated hairstyle, using an AI model to generate multiple color variations that match the hair color and skin tone, and stores the results.

[1065] Step 10:

[1066] The server compiles the generated hairstyle suggestions and coloring simulation results and sends them to the terminal, which receives the suggestions and displays them to the customer.

[1067] Step 11:

[1068] The user can check the proposed hairstyles and color variations on the device, select the style they like, and enter their feedback, which is then sent to the server.

[1069] Step 12:

[1070] The server stores the received feedback in a database and uses it for future suggestions. This feedback is also used as training data for the AI ​​model.

[1071] In this way, the entire system works together to suggest the best hairstyle for the user, and the quality of the service is continuously improved by incorporating feedback.

[1072] Example 1

[1073] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1074] Conventional hairstyle suggestion systems have the problem of being unable to fully customize hairstyles to reflect the user's requests and hair characteristics, resulting in low user satisfaction. They also lacked the functionality to provide coloring simulations and the latest trend information in real time. This made it difficult to suggest optimal hairstyles that met the diverse needs of users.

[1075] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1076] In this invention, the server includes means for receiving user image data, means for receiving the user's hairstyle requests, and means for analyzing the image data and extracting the user's hair characteristics. This makes it possible to propose customized hairstyles that are optimal for individual users by using a generation AI model means for generating an optimal hairstyle based on the user's requests and hair characteristics, and a means for proposing the generated hairstyle to the user.

[1077] "Image data" refers to photographs or image files showing the user's face or hair condition.

[1078] "Requests" refer to specific requests and wishes such as the user's desired hairstyle, hair volume, texture, length, etc.

[1079] "Hair characteristics" refers to attribute information such as hair length, volume, texture, and color obtained by analyzing image data.

[1080] A "generative AI model" refers to a system that uses machine learning algorithms to generate optimal hairstyles based on the user's requests and hair characteristics.

[1081] "Suggestion" refers to the act of displaying the generated hairstyle and coloring simulation results to the user and asking for their selection and feedback.

[1082] "Virtual coloring simulation" refers to a technology that virtually simulates different hair color variations for a generated hairstyle.

[1083] "Trend information" refers to information about the latest hairstyles and fashions, and is obtained from external data sources and APIs.

[1084] This invention is a system that efficiently proposes customized hairstyles desired by users using a beauty salon application. The system receives the user's image data and requests, analyzes them, and proposes optimal hairstyles using a generative AI model. It also makes additional suggestions based on coloring simulations and trend information.

[1085] Overall system configuration

[1086] The system consists of the following main components:

[1087] 1. Photo upload system (terminal): A user interface for users to upload their own images.

[1088] 2. Request input system (terminal): An interface for users to input desired styling elements and specific requests.

[1089] 3. Data analysis system (server): Technology for analyzing received image data and extracting the user's hair characteristics.

[1090] 4. Generation system (server): A generative AI model that generates optimal hairstyles based on the extracted hair characteristics and the user's requests.

[1091] 5. Proposal system (terminal): An interface for proposing generated hairstyles to users.

[1092] 6. Coloring Simulation System (Server): Technology that simulates virtual coloring.

[1093] 7. Trend information collection and suggestion system (server): A function that collects the latest trend information and suggests additional hairstyles based on it.

[1094] Hardware and software used

[1095] 1. Terminal: A device operated by a user, such as a smartphone, tablet, or PC.

[1096] 2. Server: A cloud server or dedicated server responsible for data analysis, processing of generative AI models, coloring simulation, and trend information collection.

[1097] 3. Software:

[1098] Image recognition technology: Uses libraries such as TensorFlow and OpenCV.

[1099] Generative AI models: Use machine learning algorithms such as GANs (generative adversarial networks).

[1100] External API: API for collecting external trend information (e.g. FashionTrendAPI, etc.).

[1101] Specific examples

[1102] Example 1: Proposing a new hairstyle to user A

[1103] 1. User uploads a photo

[1104] User A launches the application and uploads a photo of themselves. For example, they can take a frontal photo using their smartphone camera and import it into the app. User A enters their hair volume: normal, texture: straight, desired length: short, and style preference: bob.

[1105] 2. The server analyzes the data

[1106] The server uses image recognition technology to extract the hair characteristics of User A. Specifically, it uses the TensorFlow library to obtain analysis results such as hair length (medium), volume (normal), and color (brown).

[1107] 3. The server generates the hairstyle

[1108] Based on the analyzed hair characteristics and customer requests, three bob styles are generated using a generative AI model (e.g., GAN).

[1109] 4. The server acquires trend information

[1110] The server uses an external API (e.g., FashionTrendAPI) to obtain the latest hairstyle trend information, and reflects the trend of bob styles based on that information.

[1111] 5. The server runs the coloring simulation.

[1112] We use OpenCV to input data into a color model and generate four color variations (e.g., natural brown, light brown, dark brown, and reddish brown).

[1113] 6. The server sends the proposal results

[1114] The proposed hairstyle and color variation results are sent to the terminal and notified to the user.

[1115] 7. The device displays the results and receives feedback

[1116] User A checks the proposed styles, selects the one they like, and enters their feedback, which is then sent to the server.

[1117] Prompt Sentence Examples

[1118] "Based on the photo below, create hairstyle suggestions and color variations for a short bob. The hair texture is straight and the hair volume is medium."

[1119] This system allows users to receive suggestions for hairstyles that best suit their preferences, and also allows them to further customize their style based on a wide range of color variations and trend information.

[1120] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1121] Step 1: User Login and Authentication

[1122] When the user launches the app, the device displays the login screen. The user enters their username and password and presses the "Login" button. The server receives the username and password sent from the device and authenticates them by checking them against the database. The input data are the username and password, and the output is the success or failure of the authentication.

[1123] Step 2: Upload photos and enter your request

[1124] If authentication is successful, the device displays an upload screen. The user takes a photo of themselves or selects one from the gallery and uploads it. After the photo is uploaded, the device displays a request input screen. The user enters desired information such as hair volume, texture, length, and style. The device sends the uploaded photo and request data to the server. The input is the photo data and request data, and the output is the data sent to the server.

[1125] Step 3: Data analysis

[1126] The server stores the photo data and request data received from the device in a database. Next, the server analyzes the photo data using image recognition technology (e.g., TensorFlow) to extract the user's hair characteristics. For example, it analyzes information such as hair length, volume, texture, and color. The input is the photo data and request data, and the output is hair feature data.

[1127] Step 4: Hairstyle generation

[1128] The server inputs data into a generative AI model (e.g., GAN) based on the analyzed hair characteristics and the user's request. The model generates the optimal hairstyle and stores the results in an internal database. The input is hair characteristic data and request data, and the output is the generated hairstyle data.

[1129] Step 5: Collect trend information and make additional suggestions

[1130] The server accesses an external API (e.g., FashionTrendAPI) to collect the latest hairstyle trend information. It analyzes the collected trend information and reflects it in the generated hairstyle to make additional suggestions. The input is trend information, and the output is hairstyle data that reflects the trend.

[1131] Step 6: Coloring simulation

[1132] The server inputs data into a color model (e.g., OpenCV) for the generated hairstyle. The model generates multiple color variations and stores them in an internal database. The input is the generated hairstyle data, and the output is the color variation data.

[1133] Step 7: Submit your proposal

[1134] The server compiles the created hairstyle and color variation results and sends the proposal results in JSON format to the device. The device receives this and displays the proposal to the user. The input is hairstyle data and color variation data, and the output is the proposal result data.

[1135] Step 8: Viewing and Feedback on Proposal Results

[1136] The user checks the proposed hairstyles and color variations on the device, selects the style they like, and enters their feedback. The device then sends this feedback to the server. The input is the user's selection and feedback, and the output is the feedback data.

[1137] (Application example 1)

[1138] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1139] Conventional hair style suggestion systems for beauty salons are capable of generating hairstyles based on image analysis and customer requests, but it is difficult to propose these results to customers in real time. Furthermore, they lack support for customers to visually visualize the proposed style. Therefore, there is a need for a method that combines real-time performance and visual clarity, both of which are necessary for proposing hairstyles that satisfy customers.

[1140] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1141] In this invention, the server includes means for receiving image data of a customer, means for receiving a hairstyle request from the customer, means for analyzing the image data to extract characteristics of the customer's hair, means for generating an optimal hairstyle based on the customer's requests and hair characteristics, means for proposing the generated hairstyle to the customer, and means for displaying the generated hairstyle in real time using a wearable device such as smart glasses, thereby enabling visually easy-to-understand hairstyle suggestions and virtual coloring simulations to be provided to the customer in real time.

[1142] The "means for receiving image data of a customer" is an interface for inputting or uploading image data including a customer's facial photograph and hair characteristics in the beauty salon system.

[1143] The "means for receiving customer requests regarding hairstyle" is an interface for the customer to input or select styling requirements such as the desired hairstyle and coloring.

[1144] "Means for analyzing image data and extracting characteristics of a customer's hair" refers to algorithms or technologies for analyzing received image data of a customer and identifying characteristics such as hair length, volume, texture, and color.

[1145] The "means for generating the optimal hairstyle based on the customer's requests and hair characteristics" is an algorithm for designing and generating the optimal hairstyle using an AI model based on the extracted hair characteristics and customer requests.

[1146] The "means for proposing the generated hairstyle to the customer" is a display interface for visually presenting the hairstyle generated by the system to the customer and obtaining selection and feedback.

[1147] "Means for displaying the generated hairstyle in real time using a wearable device such as smart glasses" refers to technology that uses a wearable device such as smart glasses worn by a hairdresser to show the generated hairstyle to the customer in real time.

[1148] The "means for performing virtual coloring simulation" is a technology for applying multiple color variations to a generated hairstyle and visually simulating the results.

[1149] "Means of collecting the latest trend information and proposing additional hairstyles to customers based on the trends" is a function that collects the latest hairstyle trend information from external APIs and databases and makes additional suggestions to customers based on those trends.

[1150] The present invention provides a specific method for building a system for a beauty salon to suggest customized hairstyles to customers, which can display and suggest optimal hairstyles to customers in real time using a wearable device such as smart glasses.

[1151] 1. Hardware and Software Configuration

[1152] The system consists of the following major hardware and software components:

[1153] Hardware:

[1154] Smart glasses (e.g., Google Glass, Vuzix Blade)

[1155] Cloud Server

[1156] software:

[1157] Cloud-based data processing systems (e.g., AWS Lambda, Microsoft Azure)

[1158] Image recognition technology (e.g., OpenCV, Google Cloud Vision API)

[1159] Hairstyle generation system using AI models (e.g. TensorFlow, PyTorch)

[1160] Voice recognition technology (e.g., Google Speech-to-Text API)

[1161] 2. System Operation Overview

[1162] The system works as follows:

[1163] 1. Collection of customer image data

[1164] A hairdresser puts on smart glasses and uses the built-in camera to take a photo of the customer's face.

[1165] After the photo is taken, the image data is sent to a cloud server.

[1166] 2. Gathering customer requirements

[1167] The hairdresser uses voice recognition to input the customer's desired hairstyle and coloring requests.

[1168] The request data is also sent to the cloud server.

[1169] 3. Analysis of image data

[1170] The cloud server analyzes the received image data and extracts the customer's hair characteristics (length, volume, texture, color, etc.).

[1171] Image recognition technologies used include OpenCV and Google Cloud Vision API.

[1172] 4. Hairstyle generation

[1173] Based on the analyzed hair characteristics and the customer's requests, an AI model is used to generate the optimal hairstyle.

[1174] The AI ​​models used include TensorFlow and PyTorch.

[1175] 5. Virtual Coloring Simulation

[1176] A virtual coloring simulation is performed by applying multiple color variations to the generated hairstyle.

[1177] The simulation results are also stored on the cloud server.

[1178] 6. Real-time display of results

[1179] The generated hairstyle and color variation results are sent to the smart glasses.

[1180] Hairdressers make real-time suggestions to customers through smart glasses.

[1181] Specific examples

[1182] Example 1: Proposing a hairstyle for customer A

[1183] 1. The hairdresser puts on the smart glasses and takes a photo of Customer A's face.

[1184] 2. Using the voice recognition function, Customer A inputs the style he or she desires (e.g., short bob, brown color).

[1185] 3. The captured image data and requested data are sent to the cloud server.

[1186] 4. The image data is analyzed on the cloud server, and hair characteristics (e.g., medium length, straight, black hair) are extracted.

[1187] 5. Based on the analysis results and your request, the AI ​​model will generate three short bob styles and perform a virtual coloring simulation.

[1188] 6. The results are sent to the smart glasses, and the hairdresser makes suggestions to Customer A through the glasses.

[1189] 7. Customer A chooses the best style and enters feedback via voice recognition.

[1190] Prompt Sentence Examples

[1191] To ask an AI model to generate a hairstyle based on a customer's request, use the following prompt:

[1192] "Generate a suitable hairstyle based on the customer's facial photo and requests."

[1193] This process allows customers to visually see in real time which hairstyle best suits them, ensuring a highly satisfying service.

[1194] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1195] Step 1:

[1196] Input: A customer comes in and the hairdresser puts on the smart glasses.

[1197] What it does: A hairdresser uses the built-in camera in the smart glasses to take a photo of the customer's face.

[1198] Output: The captured image data is stored in the smart glasses.

[1199] Step 2:

[1200] Input: A photo of the customer's face.

[1201] Specific operation: A photo of the customer's face is sent from the smart glasses to a cloud server.

[1202] Output: The cloud server receives and stores the facial photo data.

[1203] Step 3:

[1204] Input: Customer's hairstyle and coloring requests.

[1205] Specific operation: The hairdresser uses the voice recognition function to input the customer's request by voice.

[1206] Output: The audio data is sent to a cloud server and converted into text data.

[1207] Step 4:

[1208] Input: Customer's photo data and request data.

[1209] How it works: The cloud server uses image recognition technology (e.g., OpenCV or Google Cloud Vision API) to analyze the customer's hair characteristics (length, volume, texture, color, etc.).

[1210] Output: Analysis data containing hair features is generated and saved.

[1211] Step 5:

[1212] Input: Customer's hair characteristics analysis data and request data.

[1213] What it does: A cloud server uses an AI model (e.g., TensorFlow or PyTorch) to generate the optimal hairstyle.

[1214] Output: The generated hairstyle data is saved.

[1215] Step 6:

[1216] Input: Generated hairstyle data.

[1217] Specific operation: The cloud server performs a virtual coloring simulation and applies multiple color variations.

[1218] Output: Hairstyle data including coloring simulation results is generated and saved.

[1219] Step 7:

[1220] Input: Hairstyle data including coloring simulation results.

[1221] Specific operation: The cloud server sends hairstyle data to the smart glasses.

[1222] Output: The transmitted hairstyle data is displayed on the smart glasses.

[1223] Step 8:

[1224] Input: Hairstyle data displayed on smart glasses.

[1225] How it works: A hairdresser uses smart glasses to suggest the best hairstyle and color options to a customer in real time.

[1226] Output: The customer reviews the suggested hairstyles and selects the best one.

[1227] Step 9:

[1228] Input: Customer selection and feedback.

[1229] Specific operation: The hairdresser inputs the customer's selection and feedback through voice recognition in the smart glasses.

[1230] Output: The feedback data is sent to the cloud server and stored.

[1231] This allows the server and terminal to work together, making it possible to suggest the best hairstyle for each customer in real time.

[1232] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1233] This invention is an application system for beauty salons that proposes hairstyles customized to meet customer needs, and provides more personalized services by combining it with an emotion engine that recognizes the user's emotions. This system analyzes the customer's image data, generates hairstyles according to their requests, and makes suggestions based on coloring simulations and trend information. The emotion engine also recognizes the user's emotions and optimizes the suggestions based on these.

[1234] Overall system configuration

[1235] The system consists of the following main components:

[1236] 1. Photo upload system (terminal): A user interface for customers to upload their own images.

[1237] 2. Request input system (terminal): An interface for customers to input desired styling elements and specific requests.

[1238] 3. Data analysis system (server): Technology for analyzing received image data and extracting the characteristics of the customer's hair.

[1239] 4. Generation system (server): An algorithm for generating the optimal hairstyle based on the extracted hair characteristics and customer requests.

[1240] 5. Proposal system (terminal): An interface for proposing the generated hairstyle to the customer.

[1241] 6. Coloring Simulation System (Server): Technology that simulates virtual coloring.

[1242] 7. Trend information collection and suggestion system (server): A function that collects the latest trend information and suggests additional hairstyles based on it.

[1243] 8. Emotion Engine (Server): Technology that recognizes user emotions and adjusts suggestions based on them.

[1244] Program Processing Overview

[1245] 1. Upload photos and input requests (device processing)

[1246] After launching the application and logging in, the user is presented with a photo upload screen. The user can select a photo from their photo library or take a new photo and upload it. A styling request input screen is then displayed, where the user enters information such as the desired hair volume, texture, length, and style, and the data is sent to the server.

[1247] 2. Data analysis (server processing)

[1248] The server stores the photo data and requests received from the device in a database, and then uses image recognition technology to extract the customer's hair characteristics, such as hair length, volume, texture, and color.

[1249] 3. Hairstyle generation (server processing)

[1250] The server uses an AI model to generate the optimal hairstyle based on the analyzed hair characteristics and the customer's requests, and the generated style is stored in an internal database.

[1251] 4. Collecting trend information and making additional suggestions (server processing)

[1252] The server uses external APIs and internal databases to collect the latest hairstyle trend information and provide additional hairstyle suggestions based on that information, which are also stored in the database.

[1253] 5. Coloring simulation (server processing)

[1254] The server performs a virtual coloring simulation on the generated hairstyle, using an AI model to generate and store multiple color variations that match different hair colors and skin tones.

[1255] 6. Emotion Recognition and Suggestion Adjustment (Server Processing)

[1256] The server uses an emotion engine to analyze the facial expressions and voice of the user reviewing the proposed hairstyle and recognize their emotions. Based on the recognized emotions, the server adjusts the proposal and provides personalized suggestions.

[1257] 7. Sending the proposal results (server processing)

[1258] The server compiles the generated hairstyle suggestions, coloring simulation results, and suggestions adjusted by emotion recognition, and sends them to the terminal, which receives the suggestions and displays them to the customer.

[1259] 8. Displaying and Feedback of Proposal Results (Device Processing)

[1260] The user can check the proposed hairstyles and color variations on the device, select the style they like, and enter their feedback, which is then sent to the server.

[1261] 9. Saving feedback and reflecting it in the next proposal (server processing)

[1262] The server stores the received feedback in a database and uses it for future suggestions. This feedback is also used as training data for the AI ​​model, and is used to increase user satisfaction.

[1263] Specific examples

[1264] Example 1: Proposing a new hairstyle for Customer B

[1265] 1. User uploads a photo

[1266] Customer B launches the application on the device, uploads a photo of himself, and inputs his styling requests: hair volume: thick, texture: wavy, desired length: medium, style preference: casual.

[1267] 2. The server analyzes the data

[1268] Using image recognition technology, the characteristics of Customer B's hair are extracted. The analysis reveals that the hair volume is thick, the length is medium, and the color is black.

[1269] 3. The server generates the hairstyle

[1270] Generate three casual styles based on the customer's requests and hair characteristics.

[1271] 4. The server acquires trend information

[1272] We collect the latest trend information and reflect the latest trends in casual style.

[1273] 5. The server runs the coloring simulation.

[1274] Generates four different color variations (e.g., natural black, dark brown, light brown, reddish brown).

[1275] 6. Emotion recognition and suggestion adjustment

[1276] As the user reviews the suggested styles, an emotion engine analyzes the user's facial expressions. For example, if surprise or joy is detected, the server will adjust the suggestions based on that emotion and make other suggestions as well.

[1277] 7. The server sends the proposal results

[1278] Suggested hairstyles and color variations, tailored suggestions based on emotions, are sent to the device.

[1279] 8. The device displays the results and receives feedback

[1280] Customer B reviews the proposed styles, selects the style they like, and provides feedback.

[1281] 9. The server stores the feedback

[1282] Receive feedback and incorporate it into your next proposal.

[1283] This allows Customer B to not only receive suggestions for the optimal hairstyle and coloring simulation that suits his or her preferences, but also receive more personalized suggestions based on his or her emotions, resulting in high levels of satisfaction.

[1284] The processing flow will be explained below.

[1285] Step 1:

[1286] When a user launches the application on their device, the login screen appears. The user enters their email address and password and taps the "Login" button.

[1287] Step 2:

[1288] The terminal sends login information to the server. The server checks the received login information against a database, and if authentication is successful, starts a session and responds to the terminal. If authentication fails, it returns an error message.

[1289] Step 3:

[1290] After successful login, the device will display the main screen and present the user with a photo upload screen where they can select a photo from their photo library or take a new photo and upload it.

[1291] Step 4:

[1292] Once the user selects or takes a photo, the device will then display a styling request input screen, where the user can enter the desired hair volume, texture, length, styling preferences, etc., and tap the "Send" button.

[1293] Step 5:

[1294] The device sends the uploaded photos and entered styling requests to the server, which receives this data and stores it in a database.

[1295] Step 6:

[1296] The server uses image recognition technology to analyze the received photo data and extract the customer's hair characteristics (length, volume, texture, color, etc.), which are then stored in a database.

[1297] Step 7:

[1298] The server uses an AI model to generate the optimal hairstyle based on the customer's request and the results of image analysis. The AI ​​model also takes into account past data and trend information when proposing a style.

[1299] Step 8:

[1300] The server uses external APIs and internal databases to collect the latest hairstyle trend information and makes additional hairstyle suggestions based on that information, which are also stored in the database.

[1301] Step 9:

[1302] The server then runs a virtual coloring simulation on the generated hairstyle, using an AI model to generate multiple color variations that match the hair color and skin tone, and stores the results.

[1303] Step 10:

[1304] The server uses an emotion engine to analyze the facial expressions and voice of the user confirming the proposed hairstyle and recognize their emotions. For example, emotions such as joy or dissatisfaction can be extracted from the user's facial expressions and voice.

[1305] Step 11:

[1306] The server then tailors the suggestions based on the recognized emotion, for example, by prioritizing hairstyles that indicate a happy emotion to the user.

[1307] Step 12:

[1308] The server sends the generated hairstyle suggestions, coloring simulation results, and suggestions adjusted by emotion recognition to the terminal, which receives the suggestions and displays them to the customer.

[1309] Step 13:

[1310] The user can check the proposed hairstyles and color variations on the device, select the style they like, and enter their feedback, which is then sent to the server.

[1311] Step 14:

[1312] The server stores the received feedback in a database and uses it for future suggestions. This feedback is also used as training data for the AI ​​model.

[1313] In this way, the entire system works together to suggest the best hairstyle for the user and improve the quality of service based on emotion recognition and feedback.

[1314] Example 2

[1315] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1316] While existing hair salon application systems suggest hairstyles based on customer requests, they lack personalized suggestions that take into account customer emotions and trend information. Furthermore, there is no system in place to incorporate feedback on suggested styles into future suggestions, which means customer satisfaction is not fully improved. Furthermore, coloring simulations are limited, making it difficult to fully address the diverse needs of customers.

[1317] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1318] In this invention, the server includes a means for receiving image data of a customer, a means for receiving the customer's hairstyle requests, and a means for analyzing the image data to extract the customer's hair characteristics. This makes it possible to generate and propose an optimal hairstyle based on the customer's requests and hair characteristics. The server also includes a means for recognizing the user's emotions and optimizing the proposal based on the emotions, a means for receiving feedback from the customer and reflecting it in future proposals, a means for performing a virtual coloring simulation for the generated hairstyle, and a means for collecting the latest trend information and proposing additional hairstyles based on the trends to the customer. This makes it possible to propose hairstyles that are personalized to the customer, thereby increasing customer satisfaction.

[1319] The "means for receiving image data from customers" is a function for acquiring images uploaded by customers, converting them into a format that can be used within the system, and saving them.

[1320] The "means for receiving customer requests regarding hairstyle" is a function for inputting information regarding the hairstyle desired by the customer, and collecting and saving the information.

[1321] The "means for analyzing image data and extracting characteristics of the customer's hair" is a function for identifying and extracting characteristics such as hair length, volume, texture, and color based on the received image data.

[1322] "A means for generating the optimal hairstyle based on the customer's requests and hair characteristics" is a function that uses an AI model to generate multiple candidate hairstyles based on the customer's requests and the results of data analysis.

[1323] The "means for proposing generated hairstyles to a customer" is a function for presenting generated hairstyles to a customer and encouraging them to select and evaluate them.

[1324] "Means for recognizing the user's emotions and optimizing suggestions based on them" refers to a function that analyzes the user's facial expressions and voice data and adjusts the suggestions based on their emotional state.

[1325] "Means of receiving feedback from customers and reflecting it in future proposals" is a function that records the evaluations and comments that customers make on proposals and uses them to help with future proposals.

[1326] The "means for performing a virtual coloring simulation for the generated hairstyle" is a function for simulating different hair colors for the generated hairstyle and presenting them to the customer.

[1327] "Means for collecting the latest trend information and proposing additional hairstyles based on the trends to customers" refers to a function for obtaining the latest hairstyle trend data from an external source and proposing additional hairstyles based on that data to customers.

[1328] This invention is an application system for beauty salons that proposes hairstyles customized to the customer's needs and provides more personalized services by combining it with an emotion engine that recognizes the user's emotions. The system consists of the following main components:

[1329] 1. Photo upload system (terminal)

[1330] This is a user interface for customers to upload their own images. Users launch the app on their smartphone, log in, and then select a photo from their photo library or take a new photo and upload it.

[1331] 2. Request input system (terminal)

[1332] This is the interface where the customer inputs the styling elements and specific requests they want, such as hair volume, texture, length, and style, and then sends the data to the server.

[1333] 3. Data analysis system (server)

[1334] This technology analyzes the received image data and extracts the customer's hair characteristics. This process uses the image recognition library OpenCV and the deep learning framework TensorFlow. For example, features such as hair length, volume, texture, and color can be extracted from the image.

[1335] 4. Generation System (Server)

[1336] Based on the analyzed hair characteristics and the customer's requests, an optimal hairstyle is generated using an AI model (e.g., a generative artificial network (GAN)). The generated style is saved in an internal database. An example of a prompt sentence is, "The client has a lot of hair, the texture is wavy, the desired length is medium, and the style preference is casual."

[1337] 5. Proposed system (terminal)

[1338] This is an interface for proposing generated hairstyles to customers. Users can check multiple hairstyles generated on the app and enter feedback. The feedback is sent to the server.

[1339] 6. Coloring Simulation System (Server)

[1340] A virtual coloring simulation is performed on the generated hairstyle, using an AI model to generate multiple color variations (e.g., natural black, dark brown, light brown, reddish brown) that match the hair color and skin tone, and then stored in a database.

[1341] 7. Trend information collection and proposal system (server)

[1342] This function collects the latest trend information from external APIs (e.g., SNS trend APIs) and internal databases, and then suggests additional hairstyles based on that information. Trend information is updated regularly, and new styles are suggested to customers.

[1343] 8. Emotion Engine (Server)

[1344] This technology recognizes the user's emotions and optimizes suggestions based on them. It uses the Emotion API to analyze the user's facial expressions and voice to recognize their emotional state. For example, if surprise or joy is detected, the system reevaluates the suggestions and makes personalized suggestions.

[1345] 9. Feedback Collection System (Server)

[1346] This function receives customer feedback on the proposed style and reflects it in future proposals. The received feedback is stored in a database and used as learning data for the AI ​​model.

[1347] This allows the system to not only propose optimal hairstyles based on the customer's requests and hair characteristics, but also provide personalized services that take into account the user's emotions and trend information.Furthermore, by incorporating feedback into the next proposal, customer satisfaction can be further increased.

[1348] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1349] Step 1:

[1350] User

[1351] The user launches the application and logs in. After logging in, a photo upload screen is displayed. The user selects a photo from the photo library or takes a new photo using the device's camera. The input is the user's photo data, and the output is the uploaded image data. Specifically, the user taps the smartphone app to launch it, and then enters their username and password for authentication. After authentication, the user selects or takes a photo and sends it to the server.

[1352] Step 2:

[1353] server

[1354] The server receives the uploaded image data and stores it in a database. It also analyzes it for the next step. The input is the uploaded image data, and the output is the image data stored in the database. Specifically, it receives an HTTP request and stores the image data in the database.

[1355] Step 3:

[1356] User

[1357] The user moves to the styling request input screen, enters information such as hair volume, texture, length, and style, and sends it to the server. The input is the request data entered by the user, and the output is the request data sent. Specifically, the user selects hair characteristics on the request input screen and taps the "Send" button.

[1358] Step 4:

[1359] server

[1360] The server stores the received request data in a database and analyzes it together with the image data. The input is the image data and the request data, and the output is the analyzed hair feature data. This analysis is performed using OpenCV and TensorFlow. Specifically, an image recognition algorithm is used to identify features such as hair length, volume, texture, and color.

[1361] Step 5:

[1362] server

[1363] The server uses an AI model (e.g., GAN) to generate the optimal hairstyle based on the analyzed hair features and desired data. The input is the analyzed hair feature data and desired data, and the output is the generated hairstyle data. Specifically, the server inputs a prompt statement into the AI ​​model to generate multiple hairstyles.

[1364] Step 6:

[1365] server

[1366] The server saves the generated hairstyle data in a database and sends it to the device. The input is the generated hairstyle data, and the output is the hairstyle suggestion data sent to the device. The specific operation is to save the results in a database and send them to the device as an HTTP response.

[1367] Step 7:

[1368] User

[1369] The user reviews the hairstyle suggestions generated on the device and enters feedback. The input is the proposed hairstyle data, and the output is the feedback data. Specifically, the user browses the suggested hairstyles on the app, selects them, and enters comments.

[1370] Step 8:

[1371] server

[1372] The server stores the received feedback data in a database and reflects it in the next proposal. The input is the feedback data, and the output is an updated database. Specifically, the server analyzes the feedback information and updates the database to use it in the next proposal.

[1373] Step 9:

[1374] server

[1375] The server uses an external API to collect the latest trend information. The input is the trend information obtained from the external API, and the output is the trend information stored in the database. Specifically, the server periodically sends an API request to obtain and store the latest trend information data.

[1376] Step 10:

[1377] server

[1378] The server performs a virtual coloring simulation for the generated hairstyle. The input is the generated hairstyle data and the user's skin tone data, and the output is multiple color variation data. Specifically, the server uses the coloring simulation model to generate different color variations of the hairstyle and save them in a database.

[1379] Step 11:

[1380] server

[1381] The server analyzes the user's facial expressions and voice to recognize emotions. The input is the user's facial expression and voice data, and the output is the recognized emotion data. Specifically, it calls the Emotion API, analyzes the user's facial expression and voice data, and identifies the emotion.

[1382] Step 12:

[1383] server

[1384] The server optimizes the suggestions based on the recognized emotion. The input is the recognized emotion data and the suggestion data, and the output is the suggestion data adjusted based on the emotion. Specifically, the server uses the AI ​​model to reevaluate the suggestions and reconfigure the optimal hairstyle based on the emotion.

[1385] (Application example 2)

[1386] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1387] Conventional hair salon application systems suggest hairstyles based on customer requests, but do not provide personalized suggestions based on the customer's emotions, making it difficult to maximize customer satisfaction. In addition to simply suggesting hairstyles, salons are also required to provide coloring simulations and trend information, but there is a lack of systems that provide these functions in an integrated manner. Furthermore, there is a need for a system that can incorporate real-time emotion recognition to make optimal suggestions based on the customer's emotions.

[1388] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1389] In this invention, the server includes a means for receiving image data of a customer, a means for receiving the customer's hairstyle requests, a means for analyzing the image data to extract the customer's hair characteristics, a means for generating an optimal hairstyle based on the customer's requests and hair characteristics, a means for proposing the generated hairstyle to the customer, and a means for analyzing the customer's facial expressions and voice to recognize emotions and adjust the proposed result. This makes it possible to propose an optimal hairstyle based on the customer's emotions in real time, in addition to their requests. It is also possible to perform coloring simulations and propose additional hairstyles based on the latest trend information, which can significantly improve overall customer satisfaction.

[1390] The "means for receiving customer image data" is a device that provides an interface for users to upload their photos to the system.

[1391] The "means for receiving customer requests regarding hairstyle" is a device that provides an interface through which the user can input information regarding the hairstyle and coloring they desire.

[1392] The "means for analyzing image data to extract characteristics of the customer's hair" is a device that includes an algorithm that analyzes uploaded image data and identifies characteristics such as hair length, texture, and color.

[1393] The "means for generating the optimal hairstyle based on the customer's requests and hair characteristics" is a device that integrates the user's requests with analyzed hair characteristics and generates the optimal hairstyle using an AI model.

[1394] The "means for proposing generated hairstyles to a customer" is a device that presents the hairstyles generated by the system to the user and provides an interface for receiving selections and feedback.

[1395] The "means for analyzing the customer's facial expressions and voice to recognize emotions and adjust the suggested results" refers to a device that includes an algorithm that analyzes the user's facial expressions and voice, recognizes emotions in real time, and adjusts the suggested hairstyle based on the results.

[1396] This invention relates to a system for improving customer experience in beauty salons. Specifically, the system receives customer image data and requests, analyzes them to suggest hairstyles and colorings, and recognizes emotions to adjust the suggestions, providing a more personalized service.

[1397] The system includes, among other things, the following main components:

[1398] 1. Means of receiving customer image data

[1399] It is mainly implemented in devices such as smartphones and tablets, and provides an interface for users to upload their own photos. For example, a smartphone application allows a user to take a selfie with their camera and upload it to the system. This photo data is then sent to a cloud server.

[1400] 2. A means of receiving customer requests regarding hairstyles

[1401] It provides an interface for users to input their desired hairstyle and coloring information. For example, users can input detailed requests such as hair length, texture, color, and style preferences into the app.

[1402] 3. A method for analyzing image data to extract the characteristics of a customer's hair

[1403] Using image recognition technology, the uploaded photo is analyzed to extract features such as hair length, texture, and color. This analysis can be performed using libraries such as TensorFlow and OpenCV. The analysis results are used in the next step.

[1404] 4. A means to generate optimal hairstyles based on customer requests and hair characteristics

[1405] Using a generative AI model, we generate the optimal hairstyle based on the user's request and analyzed hair characteristics. We use deep learning frameworks such as Keras and PyTorch to propose several hairstyle candidates.

[1406] 5. A means of proposing generated hairstyles to customers

[1407] The proposed hairstyles are presented to the user and an interface is provided for selection and feedback. For example, the generated hairstyles are displayed on the app, and the user can browse them and select the style they prefer.

[1408] 6. A method to recognize emotions by analyzing customer facial expressions and voice and adjust the proposal results

[1409] As users review the suggested hairstyles, the system analyzes their facial expressions and voice to recognize their emotions. This process is carried out using emotion recognition software such as EmotionEngine. Based on the recognized emotions, the system adjusts the suggestions to improve user satisfaction.

[1410] Specific examples

[1411] Let's take a specific example where customer C is looking for a new hairstyle at a hair salon.

[1412] 1. Upload a photo

[1413] Customer C opens the app, takes a photo of themselves, and uploads it.

[1414] 2. Input your request

[1415] Hair length: Medium

[1416] Hair Texture:Wave

[1417] Favorite color: Dark

[1418] 3. Data Analysis

[1419] It is analyzed as follows: Length: Medium, Texture: Wavy, Color: Black.

[1420] 4. Hairstyle Generation

[1421] Based on the analysis results, several new styles are generated.

[1422] 5. Emotion recognition

[1423] Recognize the happy expression on Customer C's face when he sees the proposed style.

[1424] 6. Final proposal result

[1425] The tailored suggestions are displayed to Customer C on her smartphone, and she chooses the style she likes best.

[1426] Prompt Sentence Examples

[1427] Image file: "path_to_your_image_file.jpg"

[1428] Customer Request:

[1429] Hair Length: Medium

[1430] Hair Texture: Wavy

[1431] Favorite color: Dark

[1432] Suggested style: "Curl"

[1433] Emotion recognition result: "Joy"

[1434] Make your final offer based on emotion.

[1435] This allows users to not only receive suggestions for the best hairstyle and coloring simulation based on their needs, but also receive more personalized suggestions based on their emotions, providing a highly satisfying experience.

[1436] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1437] Step 1:

[1438] Upload a photo

[1439] Subject: User

[1440] Description: A user launches the application, takes a photo of themselves, and uploads it to the system. The input is image data taken with the smartphone camera, and the output is data in an image file format sent to the cloud server. Specifically, the user presses the "Upload Photo" button, selects a photo, and uploads it.

[1441] Step 2:

[1442] Request input

[1443] Subject: User

[1444] Description: A user uses an application interface to input detailed information about their desired hairstyle and color. The input is the user's style preferences provided through text and options, and the output is data about those preferences sent to the server. Specific operations include selecting length, texture, color, etc. on the "Enter Hairstyle Preferences" screen and submitting it.

[1445] Step 3:

[1446] Data analysis

[1447] Subject: Server

[1448] Description: The server analyzes and processes the image data received and extracts the user's hair features. The input is the uploaded image data, and the output is the analysis results such as hair length, texture, and color. Specifically, it uses TensorFlow and OpenCV to detect hair areas from the image and extracts their features as numerical data.

[1449] Step 4:

[1450] Hairstyle generation

[1451] Subject: Server

[1452] Description: The server uses a generative AI model to generate the optimal hairstyle based on the analyzed hair features and the user's requests. The input is hair feature data and user request data, and the output is generated hairstyle candidates. Specifically, it applies a trained model using Keras or PyTorch to generate several optimal hairstyles.

[1453] Step 5:

[1454] Providing proposal results

[1455] Subject: Server

[1456] Description: The generated hairstyle is proposed to the user and made available for viewing. The input is the data of the generated hairstyle, and the output is the proposal information sent to the user's device. Specifically, the application generates images of the multiple generated hairstyles and displays them on the application screen.

[1457] Step 6:

[1458] emotion recognition

[1459] Subject: Server

[1460] Description: The server analyzes the user's facial expressions and voice to recognize emotions. The input is the user's facial expression and voice data, and the output is the recognized emotion data. Specifically, it uses the Emotion Engine to analyze data obtained from the camera and microphone and quantifies the user's emotions.

[1461] Step 7:

[1462] Adjustment of proposed results

[1463] Subject: Server

[1464] Description: Adjusts suggested hairstyles based on emotion recognition results. The input is the recognized emotion data and initial hairstyle suggestion data, and the output is the adjusted final suggestion data. Specific operations include modifying suggested hairstyles or making additional suggestions based on the recognized emotion.

[1465] Step 8:

[1466] Provision of final proposal results

[1467] Subject: Server

[1468] Description: Provides the user with a final hairstyle proposal after adjustments. The input is the adjusted proposal data, and the output is the final proposal information sent to the user's device. Specific operations include displaying the adjusted hairstyle proposal in the application so that the user can confirm it.

[1469] Step 9:

[1470] Collecting and storing feedback

[1471] Subject: User and Server

[1472] Description: The user reviews the suggested styles, selects the one they like, and provides feedback. This feedback is sent to the server and saved. The input is the hairstyle selected by the user and the feedback data, and the output is the feedback data saved on the server. Specifically, when the user presses the "Send Feedback" button, the information entered is sent to the server and used for the next suggestion.

[1473] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1474] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1475] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[1476] [Fourth embodiment]

[1477] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[1478] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1479] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1480] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[1481] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1482] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1483] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1484] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[1485] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1486] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1487] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1488] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1489] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1490] This invention is a system for building an application for beauty salons that proposes customized hairstyles that meet customer needs. This system analyzes the customer's image data, generates the optimal hairstyle, and provides coloring simulations and the latest trend information.

[1491] Overall system configuration

[1492] The system consists of the following main components:

[1493] 1. Photo upload system (terminal): A user interface for customers to upload their own images.

[1494] 2. Request input system (terminal): An interface for customers to input desired styling elements and specific requests.

[1495] 3. Data analysis system (server): Technology for analyzing received image data and extracting the characteristics of the customer's hair.

[1496] 4. Generation system (server): An algorithm for generating the optimal hairstyle based on the extracted hair characteristics and customer requests.

[1497] 5. Proposal system (terminal): An interface for proposing the generated hairstyle to the customer.

[1498] 6. Coloring Simulation System (Server): Technology that simulates virtual coloring.

[1499] 7. Trend information collection and suggestion system (server): A function that collects the latest trend information and suggests additional hairstyles based on it.

[1500] Program Processing Overview

[1501] 1. Upload photos and input requests (device processing)

[1502] The customer launches the application and logs in. After logging in, a photo upload screen appears, where the customer can take or select a photo of themselves. Next, a screen appears where the customer can enter information such as the amount, texture, length, and style of their desired hair. This data is then sent from the device to the server.

[1503] 2. Data analysis (server processing)

[1504] The server stores the photo data and requests received from the device in a database, and then uses image recognition technology to extract the customer's hair characteristics, such as hair length, volume, texture, and color.

[1505] 3. Hairstyle generation (server processing)

[1506] The server uses an AI model to generate the optimal hairstyle based on the analyzed hair characteristics and the customer's requests, and the generated style is stored in an internal database.

[1507] 4. Collecting trend information and making additional suggestions (server processing)

[1508] The server uses external APIs and internal databases to collect the latest hairstyle trends, and then provides additional hairstyle suggestions based on these trends.

[1509] 5. Coloring simulation (server processing)

[1510] The server then performs a virtual coloring simulation on the generated hairstyle, using an AI model to generate and save color variations that match the hair color and skin tone.

[1511] 6. Sending the proposal results (server processing)

[1512] The server sends the hairstyle proposal results and coloring simulation results to the terminal, which receives them and displays them to the customer.

[1513] 7. Displaying and Feedback of Proposal Results (Device Processing)

[1514] The customer checks the proposed hairstyles and color variations on the terminal and inputs their selection or feedback, which is then sent to the server.

[1515] Specific examples

[1516] Example 1: Proposing a new hairstyle to Customer A

[1517] 1. User uploads a photo

[1518] Customer A launches the application on the device, uploads a photo of himself, and inputs his hair volume: normal, texture: straight, desired length: short, and style preference: bob.

[1519] 2. The server analyzes the data

[1520] Using image recognition technology, the characteristics of Customer A's hair are extracted. The analysis reveals that hair volume is normal, length is medium, and color is brown.

[1521] 3. The server generates the hairstyle

[1522] Generate three bob styles based on the client's requests and hair characteristics.

[1523] 4. The server acquires trend information

[1524] We collect the latest trend information and reflect the trends in bob styles from that information.

[1525] 5. The server runs the coloring simulation.

[1526] Generates four color variations (e.g., natural brown, light brown, dark brown, reddish brown).

[1527] 6. The server sends the proposal results

[1528] Suggested hairstyles and color variations are sent to your device.

[1529] 7. The device displays the results and receives feedback

[1530] Customer A reviews the proposed styles, selects the style they like, and enters their feedback.

[1531] This allows customer A to receive suggestions for the best hairstyle that suits his or her desires.

[1532] The processing flow will be explained below.

[1533] Step 1:

[1534] When a user launches the application on their device, the login screen appears. The user enters their email address and password and taps the "Login" button.

[1535] Step 2:

[1536] The terminal sends login information to the server. The server checks the received login information against a database, and if authentication is successful, starts a session and responds to the terminal. If authentication fails, it returns an error message.

[1537] Step 3:

[1538] After successful login, the device will display the main screen and present the user with a photo upload screen where they can select a photo from their photo library or take a new photo and upload it.

[1539] Step 4:

[1540] Once the user selects or takes a photo, the device will then display a styling request input screen, where the user can enter the desired hair volume, texture, length, styling preferences, etc., and tap the "Send" button.

[1541] Step 5:

[1542] The device sends the uploaded photos and entered styling requests to the server, which receives this data and stores it in a database.

[1543] Step 6:

[1544] The server uses image recognition technology to analyze the received photo data and extract the customer's hair characteristics (length, volume, texture, color, etc.), which are then stored in a database.

[1545] Step 7:

[1546] The server uses an AI model to generate the optimal hairstyle based on the customer's request and the results of image analysis. The AI ​​model also takes into account past data and trend information when proposing a style.

[1547] Step 8:

[1548] The server uses external APIs and internal databases to collect the latest hairstyle trend information and makes additional hairstyle suggestions based on that information, which are also stored in the database.

[1549] Step 9:

[1550] The server then runs a virtual coloring simulation on the generated hairstyle, using an AI model to generate multiple color variations that match the hair color and skin tone, and stores the results.

[1551] Step 10:

[1552] The server compiles the generated hairstyle suggestions and coloring simulation results and sends them to the terminal, which receives the suggestions and displays them to the customer.

[1553] Step 11:

[1554] The user can check the proposed hairstyles and color variations on the device, select the style they like, and enter their feedback, which is then sent to the server.

[1555] Step 12:

[1556] The server stores the received feedback in a database and uses it for future suggestions. This feedback is also used as training data for the AI ​​model.

[1557] In this way, the entire system works together to suggest the best hairstyle for the user, and the quality of the service is continuously improved by incorporating feedback.

[1558] Example 1

[1559] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1560] Conventional hairstyle suggestion systems have the problem of being unable to fully customize hairstyles to reflect the user's requests and hair characteristics, resulting in low user satisfaction. They also lacked the functionality to provide coloring simulations and the latest trend information in real time. This made it difficult to suggest optimal hairstyles that met the diverse needs of users.

[1561] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1562] In this invention, the server includes means for receiving user image data, means for receiving the user's hairstyle requests, and means for analyzing the image data and extracting the user's hair characteristics. This makes it possible to propose customized hairstyles that are optimal for individual users by using a generation AI model means for generating an optimal hairstyle based on the user's requests and hair characteristics, and a means for proposing the generated hairstyle to the user.

[1563] "Image data" refers to photographs or image files showing the user's face or hair condition.

[1564] "Requests" refer to specific requests and wishes such as the user's desired hairstyle, hair volume, texture, length, etc.

[1565] "Hair characteristics" refers to attribute information such as hair length, volume, texture, and color obtained by analyzing image data.

[1566] A "generative AI model" refers to a system that uses machine learning algorithms to generate optimal hairstyles based on the user's requests and hair characteristics.

[1567] "Suggestion" refers to the act of displaying the generated hairstyle and coloring simulation results to the user and asking for their selection and feedback.

[1568] "Virtual coloring simulation" refers to a technology that virtually simulates different hair color variations for a generated hairstyle.

[1569] "Trend information" refers to information about the latest hairstyles and fashions, and is obtained from external data sources and APIs.

[1570] This invention is a system that efficiently proposes customized hairstyles desired by users using a beauty salon application. The system receives the user's image data and requests, analyzes them, and proposes optimal hairstyles using a generative AI model. It also makes additional suggestions based on coloring simulations and trend information.

[1571] Overall system configuration

[1572] The system consists of the following main components:

[1573] 1. Photo upload system (terminal): A user interface for users to upload their own images.

[1574] 2. Request input system (terminal): An interface for users to input desired styling elements and specific requests.

[1575] 3. Data analysis system (server): Technology for analyzing received image data and extracting the user's hair characteristics.

[1576] 4. Generation system (server): A generative AI model that generates optimal hairstyles based on the extracted hair characteristics and the user's requests.

[1577] 5. Proposal system (terminal): An interface for proposing generated hairstyles to users.

[1578] 6. Coloring Simulation System (Server): Technology that simulates virtual coloring.

[1579] 7. Trend information collection and suggestion system (server): A function that collects the latest trend information and suggests additional hairstyles based on it.

[1580] Hardware and software used

[1581] 1. Terminal: A device operated by a user, such as a smartphone, tablet, or PC.

[1582] 2. Server: A cloud server or dedicated server responsible for data analysis, processing of generative AI models, coloring simulation, and trend information collection.

[1583] 3. Software:

[1584] Image recognition technology: Uses libraries such as TensorFlow and OpenCV.

[1585] Generative AI models: Use machine learning algorithms such as GANs (generative adversarial networks).

[1586] External API: API for collecting external trend information (e.g. FashionTrendAPI, etc.).

[1587] Specific examples

[1588] Example 1: Proposing a new hairstyle to user A

[1589] 1. User uploads a photo

[1590] User A launches the application and uploads a photo of themselves. For example, they can take a frontal photo using their smartphone camera and import it into the app. User A enters their hair volume: normal, texture: straight, desired length: short, and style preference: bob.

[1591] 2. The server analyzes the data

[1592] The server uses image recognition technology to extract the hair characteristics of User A. Specifically, it uses the TensorFlow library to obtain analysis results such as hair length (medium), volume (normal), and color (brown).

[1593] 3. The server generates the hairstyle

[1594] Based on the analyzed hair characteristics and customer requests, three bob styles are generated using a generative AI model (e.g., GAN).

[1595] 4. The server acquires trend information

[1596] The server uses an external API (e.g., FashionTrendAPI) to obtain the latest hairstyle trend information, and reflects the trend of bob styles based on that information.

[1597] 5. The server runs the coloring simulation.

[1598] We use OpenCV to input data into a color model and generate four color variations (e.g., natural brown, light brown, dark brown, and reddish brown).

[1599] 6. The server sends the proposal results

[1600] The proposed hairstyle and color variation results are sent to the terminal and notified to the user.

[1601] 7. The device displays the results and receives feedback

[1602] User A checks the proposed styles, selects the one they like, and enters their feedback, which is then sent to the server.

[1603] Prompt Sentence Examples

[1604] "Based on the photo below, create hairstyle suggestions and color variations for a short bob. The hair texture is straight and the hair volume is medium."

[1605] This system allows users to receive suggestions for hairstyles that best suit their preferences, and also allows them to further customize their style based on a wide range of color variations and trend information.

[1606] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1607] Step 1: User Login and Authentication

[1608] When the user launches the app, the device displays the login screen. The user enters their username and password and presses the "Login" button. The server receives the username and password sent from the device and authenticates them by checking them against the database. The input data are the username and password, and the output is the success or failure of the authentication.

[1609] Step 2: Upload photos and enter your request

[1610] If authentication is successful, the device displays an upload screen. The user takes a photo of themselves or selects one from the gallery and uploads it. After the photo is uploaded, the device displays a request input screen. The user enters desired information such as hair volume, texture, length, and style. The device sends the uploaded photo and request data to the server. The input is the photo data and request data, and the output is the data sent to the server.

[1611] Step 3: Data analysis

[1612] The server stores the photo data and request data received from the device in a database. Next, the server analyzes the photo data using image recognition technology (e.g., TensorFlow) to extract the user's hair characteristics. For example, it analyzes information such as hair length, volume, texture, and color. The input is the photo data and request data, and the output is hair feature data.

[1613] Step 4: Hairstyle generation

[1614] The server inputs data into a generative AI model (e.g., GAN) based on the analyzed hair characteristics and the user's request. The model generates the optimal hairstyle and stores the results in an internal database. The input is hair characteristic data and request data, and the output is the generated hairstyle data.

[1615] Step 5: Collect trend information and make additional suggestions

[1616] The server accesses an external API (e.g., FashionTrendAPI) to collect the latest hairstyle trend information. It analyzes the collected trend information and reflects it in the generated hairstyle to make additional suggestions. The input is trend information, and the output is hairstyle data that reflects the trend.

[1617] Step 6: Coloring simulation

[1618] The server inputs data into a color model (e.g., OpenCV) for the generated hairstyle. The model generates multiple color variations and stores them in an internal database. The input is the generated hairstyle data, and the output is the color variation data.

[1619] Step 7: Submit your proposal

[1620] The server compiles the created hairstyle and color variation results and sends the proposal results in JSON format to the device. The device receives this and displays the proposal to the user. The input is hairstyle data and color variation data, and the output is the proposal result data.

[1621] Step 8: Viewing and Feedback on Proposal Results

[1622] The user checks the proposed hairstyles and color variations on the device, selects the style they like, and enters their feedback. The device then sends this feedback to the server. The input is the user's selection and feedback, and the output is the feedback data.

[1623] (Application example 1)

[1624] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1625] Conventional hair style suggestion systems for beauty salons are capable of generating hairstyles based on image analysis and customer requests, but it is difficult to propose these results to customers in real time. Furthermore, they lack support for customers to visually visualize the proposed style. Therefore, there is a need for a method that combines real-time performance and visual clarity, both of which are necessary for proposing hairstyles that satisfy customers.

[1626] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1627] In this invention, the server includes means for receiving image data of a customer, means for receiving a hairstyle request from the customer, means for analyzing the image data to extract characteristics of the customer's hair, means for generating an optimal hairstyle based on the customer's requests and hair characteristics, means for proposing the generated hairstyle to the customer, and means for displaying the generated hairstyle in real time using a wearable device such as smart glasses, thereby enabling visually easy-to-understand hairstyle suggestions and virtual coloring simulations to be provided to the customer in real time.

[1628] The "means for receiving image data of a customer" is an interface for inputting or uploading image data including a customer's facial photograph and hair characteristics in the beauty salon system.

[1629] The "means for receiving customer requests regarding hairstyle" is an interface for the customer to input or select styling requirements such as the desired hairstyle and coloring.

[1630] "Means for analyzing image data and extracting characteristics of a customer's hair" refers to algorithms or technologies for analyzing received image data of a customer and identifying characteristics such as hair length, volume, texture, and color.

[1631] The "means for generating the optimal hairstyle based on the customer's requests and hair characteristics" is an algorithm for designing and generating the optimal hairstyle using an AI model based on the extracted hair characteristics and customer requests.

[1632] The "means for proposing the generated hairstyle to the customer" is a display interface for visually presenting the hairstyle generated by the system to the customer and obtaining selection and feedback.

[1633] "Means for displaying the generated hairstyle in real time using a wearable device such as smart glasses" refers to technology that uses a wearable device such as smart glasses worn by a hairdresser to show the generated hairstyle to the customer in real time.

[1634] The "means for performing virtual coloring simulation" is a technology for applying multiple color variations to a generated hairstyle and visually simulating the results.

[1635] "Means of collecting the latest trend information and proposing additional hairstyles to customers based on the trends" is a function that collects the latest hairstyle trend information from external APIs and databases and makes additional suggestions to customers based on those trends.

[1636] The present invention provides a specific method for building a system for a beauty salon to suggest customized hairstyles to customers, which can display and suggest optimal hairstyles to customers in real time using a wearable device such as smart glasses.

[1637] 1. Hardware and Software Configuration

[1638] The system consists of the following major hardware and software components:

[1639] Hardware:

[1640] Smart glasses (e.g., Google Glass, Vuzix Blade)

[1641] Cloud Server

[1642] software:

[1643] Cloud-based data processing systems (e.g., AWS Lambda, Microsoft Azure)

[1644] Image recognition technology (e.g., OpenCV, Google Cloud Vision API)

[1645] Hairstyle generation system using AI models (e.g. TensorFlow, PyTorch)

[1646] Voice recognition technology (e.g., Google Speech-to-Text API)

[1647] 2. System Operation Overview

[1648] The system works as follows:

[1649] 1. Collection of customer image data

[1650] A hairdresser puts on smart glasses and uses the built-in camera to take a photo of the customer's face.

[1651] After the photo is taken, the image data is sent to a cloud server.

[1652] 2. Gathering customer requirements

[1653] The hairdresser uses voice recognition to input the customer's desired hairstyle and coloring requests.

[1654] The request data is also sent to the cloud server.

[1655] 3. Analysis of image data

[1656] The cloud server analyzes the received image data and extracts the customer's hair characteristics (length, volume, texture, color, etc.).

[1657] Image recognition technologies used include OpenCV and Google Cloud Vision API.

[1658] 4. Hairstyle generation

[1659] Based on the analyzed hair characteristics and the customer's requests, an AI model is used to generate the optimal hairstyle.

[1660] The AI ​​models used include TensorFlow and PyTorch.

[1661] 5. Virtual Coloring Simulation

[1662] A virtual coloring simulation is performed by applying multiple color variations to the generated hairstyle.

[1663] The simulation results are also stored on the cloud server.

[1664] 6. Real-time display of results

[1665] The generated hairstyle and color variation results are sent to the smart glasses.

[1666] Hairdressers make real-time suggestions to customers through smart glasses.

[1667] Specific examples

[1668] Example 1: Proposing a hairstyle for customer A

[1669] 1. The hairdresser puts on the smart glasses and takes a photo of Customer A's face.

[1670] 2. Using the voice recognition function, Customer A inputs the style he or she desires (e.g., short bob, brown color).

[1671] 3. The captured image data and requested data are sent to the cloud server.

[1672] 4. The image data is analyzed on the cloud server, and hair characteristics (e.g., medium length, straight, black hair) are extracted.

[1673] 5. Based on the analysis results and your request, the AI ​​model will generate three short bob styles and perform a virtual coloring simulation.

[1674] 6. The results are sent to the smart glasses, and the hairdresser makes suggestions to Customer A through the glasses.

[1675] 7. Customer A chooses the best style and enters feedback via voice recognition.

[1676] Prompt Sentence Examples

[1677] To ask an AI model to generate a hairstyle based on a customer's request, use the following prompt:

[1678] "Generate a suitable hairstyle based on the customer's facial photo and requests."

[1679] This process allows customers to visually see in real time which hairstyle best suits them, ensuring a highly satisfying service.

[1680] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1681] Step 1:

[1682] Input: A customer comes in and the hairdresser puts on the smart glasses.

[1683] What it does: A hairdresser uses the built-in camera in the smart glasses to take a photo of the customer's face.

[1684] Output: The captured image data is stored in the smart glasses.

[1685] Step 2:

[1686] Input: A photo of the customer's face.

[1687] Specific operation: A photo of the customer's face is sent from the smart glasses to a cloud server.

[1688] Output: The cloud server receives and stores the facial photo data.

[1689] Step 3:

[1690] Input: Customer's hairstyle and coloring requests.

[1691] Specific operation: The hairdresser uses the voice recognition function to input the customer's request by voice.

[1692] Output: The audio data is sent to a cloud server and converted into text data.

[1693] Step 4:

[1694] Input: Customer's photo data and request data.

[1695] How it works: The cloud server uses image recognition technology (e.g., OpenCV or Google Cloud Vision API) to analyze the customer's hair characteristics (length, volume, texture, color, etc.).

[1696] Output: Analysis data containing hair features is generated and saved.

[1697] Step 5:

[1698] Input: Customer's hair characteristics analysis data and request data.

[1699] What it does: A cloud server uses an AI model (e.g., TensorFlow or PyTorch) to generate the optimal hairstyle.

[1700] Output: The generated hairstyle data is saved.

[1701] Step 6:

[1702] Input: Generated hairstyle data.

[1703] Specific operation: The cloud server performs a virtual coloring simulation and applies multiple color variations.

[1704] Output: Hairstyle data including coloring simulation results is generated and saved.

[1705] Step 7:

[1706] Input: Hairstyle data including coloring simulation results.

[1707] Specific operation: The cloud server sends hairstyle data to the smart glasses.

[1708] Output: The transmitted hairstyle data is displayed on the smart glasses.

[1709] Step 8:

[1710] Input: Hairstyle data displayed on smart glasses.

[1711] How it works: A hairdresser uses smart glasses to suggest the best hairstyle and color options to a customer in real time.

[1712] Output: The customer reviews the suggested hairstyles and selects the best one.

[1713] Step 9:

[1714] Input: Customer selection and feedback.

[1715] Specific operation: The hairdresser inputs the customer's selection and feedback through voice recognition in the smart glasses.

[1716] Output: The feedback data is sent to the cloud server and stored.

[1717] This allows the server and terminal to work together, making it possible to suggest the best hairstyle for each customer in real time.

[1718] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1719] This invention is an application system for beauty salons that proposes hairstyles customized to meet customer needs, and provides more personalized services by combining it with an emotion engine that recognizes the user's emotions. This system analyzes the customer's image data, generates hairstyles according to their requests, and makes suggestions based on coloring simulations and trend information. The emotion engine also recognizes the user's emotions and optimizes the suggestions based on these.

[1720] Overall system configuration

[1721] The system consists of the following main components:

[1722] 1. Photo upload system (terminal): A user interface for customers to upload their own images.

[1723] 2. Request input system (terminal): An interface for customers to input desired styling elements and specific requests.

[1724] 3. Data analysis system (server): Technology for analyzing received image data and extracting the characteristics of the customer's hair.

[1725] 4. Generation system (server): An algorithm for generating the optimal hairstyle based on the extracted hair characteristics and customer requests.

[1726] 5. Proposal system (terminal): An interface for proposing the generated hairstyle to the customer.

[1727] 6. Coloring Simulation System (Server): Technology that simulates virtual coloring.

[1728] 7. Trend information collection and suggestion system (server): A function that collects the latest trend information and suggests additional hairstyles based on it.

[1729] 8. Emotion Engine (Server): Technology that recognizes user emotions and adjusts suggestions based on them.

[1730] Program Processing Overview

[1731] 1. Upload photos and input requests (device processing)

[1732] After launching the application and logging in, the user is presented with a photo upload screen. The user can select a photo from their photo library or take a new photo and upload it. A styling request input screen is then displayed, where the user enters information such as the desired hair volume, texture, length, and style, and the data is sent to the server.

[1733] 2. Data analysis (server processing)

[1734] The server stores the photo data and requests received from the device in a database, and then uses image recognition technology to extract the customer's hair characteristics, such as hair length, volume, texture, and color.

[1735] 3. Hairstyle generation (server processing)

[1736] The server uses an AI model to generate the optimal hairstyle based on the analyzed hair characteristics and the customer's requests, and the generated style is stored in an internal database.

[1737] 4. Collecting trend information and making additional suggestions (server processing)

[1738] The server uses external APIs and internal databases to collect the latest hairstyle trend information and provide additional hairstyle suggestions based on that information, which are also stored in the database.

[1739] 5. Coloring simulation (server processing)

[1740] The server performs a virtual coloring simulation on the generated hairstyle, using an AI model to generate and store multiple color variations that match different hair colors and skin tones.

[1741] 6. Emotion Recognition and Suggestion Adjustment (Server Processing)

[1742] The server uses an emotion engine to analyze the facial expressions and voice of the user reviewing the proposed hairstyle and recognize their emotions. Based on the recognized emotions, the server adjusts the proposal and provides personalized suggestions.

[1743] 7. Sending the proposal results (server processing)

[1744] The server compiles the generated hairstyle suggestions, coloring simulation results, and suggestions adjusted by emotion recognition, and sends them to the terminal, which receives the suggestions and displays them to the customer.

[1745] 8. Displaying and Feedback of Proposal Results (Device Processing)

[1746] The user can check the proposed hairstyles and color variations on the device, select the style they like, and enter their feedback, which is then sent to the server.

[1747] 9. Saving feedback and reflecting it in the next proposal (server processing)

[1748] The server stores the received feedback in a database and uses it for future suggestions. This feedback is also used as training data for the AI ​​model, and is used to increase user satisfaction.

[1749] Specific examples

[1750] Example 1: Proposing a new hairstyle for Customer B

[1751] 1. User uploads a photo

[1752] Customer B launches the application on the device, uploads a photo of himself, and inputs his styling requests: hair volume: thick, texture: wavy, desired length: medium, style preference: casual.

[1753] 2. The server analyzes the data

[1754] Using image recognition technology, the characteristics of Customer B's hair are extracted. The analysis reveals that the hair volume is thick, the length is medium, and the color is black.

[1755] 3. The server generates the hairstyle

[1756] Generate three casual styles based on the customer's requests and hair characteristics.

[1757] 4. The server acquires trend information

[1758] We collect the latest trend information and reflect the latest trends in casual style.

[1759] 5. The server runs the coloring simulation.

[1760] Generates four different color variations (e.g., natural black, dark brown, light brown, reddish brown).

[1761] 6. Emotion recognition and suggestion adjustment

[1762] As the user reviews the suggested styles, an emotion engine analyzes the user's facial expressions. For example, if surprise or joy is detected, the server will adjust the suggestions based on that emotion and make other suggestions as well.

[1763] 7. The server sends the proposal results

[1764] Suggested hairstyles and color variations, tailored suggestions based on emotions, are sent to the device.

[1765] 8. The device displays the results and receives feedback

[1766] Customer B reviews the proposed styles, selects the style they like, and provides feedback.

[1767] 9. The server stores the feedback

[1768] Receive feedback and incorporate it into your next proposal.

[1769] This allows Customer B to not only receive suggestions for the optimal hairstyle and coloring simulation that suits his or her preferences, but also receive more personalized suggestions based on his or her emotions, resulting in high levels of satisfaction.

[1770] The processing flow will be explained below.

[1771] Step 1:

[1772] When a user launches the application on their device, the login screen appears. The user enters their email address and password and taps the "Login" button.

[1773] Step 2:

[1774] The terminal sends login information to the server. The server checks the received login information against a database, and if authentication is successful, starts a session and responds to the terminal. If authentication fails, it returns an error message.

[1775] Step 3:

[1776] After successful login, the device will display the main screen and present the user with a photo upload screen where they can select a photo from their photo library or take a new photo and upload it.

[1777] Step 4:

[1778] Once the user selects or takes a photo, the device will then display a styling request input screen, where the user can enter the desired hair volume, texture, length, styling preferences, etc., and tap the "Send" button.

[1779] Step 5:

[1780] The device sends the uploaded photos and entered styling requests to the server, which receives this data and stores it in a database.

[1781] Step 6:

[1782] The server uses image recognition technology to analyze the received photo data and extract the customer's hair characteristics (length, volume, texture, color, etc.), which are then stored in a database.

[1783] Step 7:

[1784] The server uses an AI model to generate the optimal hairstyle based on the customer's request and the results of image analysis. The AI ​​model also takes into account past data and trend information when proposing a style.

[1785] Step 8:

[1786] The server uses external APIs and internal databases to collect the latest hairstyle trend information and makes additional hairstyle suggestions based on that information, which are also stored in the database.

[1787] Step 9:

[1788] The server then runs a virtual coloring simulation on the generated hairstyle, using an AI model to generate multiple color variations that match the hair color and skin tone, and stores the results.

[1789] Step 10:

[1790] The server uses an emotion engine to analyze the facial expressions and voice of the user confirming the proposed hairstyle and recognize their emotions. For example, emotions such as joy or dissatisfaction can be extracted from the user's facial expressions and voice.

[1791] Step 11:

[1792] The server then tailors the suggestions based on the recognized emotion, for example, by prioritizing hairstyles that indicate a happy emotion to the user.

[1793] Step 12:

[1794] The server sends the generated hairstyle suggestions, coloring simulation results, and suggestions adjusted by emotion recognition to the terminal, which receives the suggestions and displays them to the customer.

[1795] Step 13:

[1796] The user can check the proposed hairstyles and color variations on the device, select the style they like, and enter their feedback, which is then sent to the server.

[1797] Step 14:

[1798] The server stores the received feedback in a database and uses it for future suggestions. This feedback is also used as training data for the AI ​​model.

[1799] In this way, the entire system works together to suggest the best hairstyle for the user and improve the quality of service based on emotion recognition and feedback.

[1800] Example 2

[1801] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1802] While existing hair salon application systems suggest hairstyles based on customer requests, they lack personalized suggestions that take into account customer emotions and trend information. Furthermore, there is no system in place to incorporate feedback on suggested styles into future suggestions, which means customer satisfaction is not fully improved. Furthermore, coloring simulations are limited, making it difficult to fully address the diverse needs of customers.

[1803] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1804] In this invention, the server includes a means for receiving image data of a customer, a means for receiving the customer's hairstyle requests, and a means for analyzing the image data to extract the customer's hair characteristics. This makes it possible to generate and propose an optimal hairstyle based on the customer's requests and hair characteristics. The server also includes a means for recognizing the user's emotions and optimizing the proposal based on the emotions, a means for receiving feedback from the customer and reflecting it in future proposals, a means for performing a virtual coloring simulation for the generated hairstyle, and a means for collecting the latest trend information and proposing additional hairstyles based on the trends to the customer. This makes it possible to propose hairstyles that are personalized to the customer, thereby increasing customer satisfaction.

[1805] The "means for receiving image data from customers" is a function for acquiring images uploaded by customers, converting them into a format that can be used within the system, and saving them.

[1806] The "means for receiving customer requests regarding hairstyle" is a function for inputting information regarding the hairstyle desired by the customer, and collecting and saving the information.

[1807] The "means for analyzing image data and extracting characteristics of the customer's hair" is a function for identifying and extracting characteristics such as hair length, volume, texture, and color based on the received image data.

[1808] "A means for generating the optimal hairstyle based on the customer's requests and hair characteristics" is a function that uses an AI model to generate multiple candidate hairstyles based on the customer's requests and the results of data analysis.

[1809] The "means for proposing generated hairstyles to a customer" is a function for presenting generated hairstyles to a customer and encouraging them to select and evaluate them.

[1810] "Means for recognizing the user's emotions and optimizing suggestions based on them" refers to a function that analyzes the user's facial expressions and voice data and adjusts the suggestions based on their emotional state.

[1811] "Means of receiving feedback from customers and reflecting it in future proposals" is a function that records the evaluations and comments that customers make on proposals and uses them to help with future proposals.

[1812] The "means for performing a virtual coloring simulation for the generated hairstyle" is a function for simulating different hair colors for the generated hairstyle and presenting them to the customer.

[1813] "Means for collecting the latest trend information and proposing additional hairstyles based on the trends to customers" refers to a function for obtaining the latest hairstyle trend data from an external source and proposing additional hairstyles based on that data to customers.

[1814] This invention is an application system for beauty salons that proposes hairstyles customized to the customer's needs and provides more personalized services by combining it with an emotion engine that recognizes the user's emotions. The system consists of the following main components:

[1815] 1. Photo upload system (terminal)

[1816] This is a user interface for customers to upload their own images. Users launch the app on their smartphone, log in, and then select a photo from their photo library or take a new photo and upload it.

[1817] 2. Request input system (terminal)

[1818] This is the interface where the customer inputs the styling elements and specific requests they want, such as hair volume, texture, length, and style, and then sends the data to the server.

[1819] 3. Data analysis system (server)

[1820] This technology analyzes the received image data and extracts the customer's hair characteristics. This process uses the image recognition library OpenCV and the deep learning framework TensorFlow. For example, features such as hair length, volume, texture, and color can be extracted from the image.

[1821] 4. Generation System (Server)

[1822] Based on the analyzed hair characteristics and the customer's requests, an optimal hairstyle is generated using an AI model (e.g., a generative artificial network (GAN)). The generated style is saved in an internal database. An example of a prompt sentence is, "The client has a lot of hair, the texture is wavy, the desired length is medium, and the style preference is casual."

[1823] 5. Proposed system (terminal)

[1824] This is an interface for proposing generated hairstyles to customers. Users can check multiple hairstyles generated on the app and enter feedback. The feedback is sent to the server.

[1825] 6. Coloring Simulation System (Server)

[1826] A virtual coloring simulation is performed on the generated hairstyle, using an AI model to generate multiple color variations (e.g., natural black, dark brown, light brown, reddish brown) that match the hair color and skin tone, and then stored in a database.

[1827] 7. Trend information collection and proposal system (server)

[1828] This function collects the latest trend information from external APIs (e.g., SNS trend APIs) and internal databases, and then suggests additional hairstyles based on that information. Trend information is updated regularly, and new styles are suggested to customers.

[1829] 8. Emotion Engine (Server)

[1830] This technology recognizes the user's emotions and optimizes suggestions based on them. It uses the Emotion API to analyze the user's facial expressions and voice to recognize their emotional state. For example, if surprise or joy is detected, the system reevaluates the suggestions and makes personalized suggestions.

[1831] 9. Feedback Collection System (Server)

[1832] This function receives customer feedback on the proposed style and reflects it in future proposals. The received feedback is stored in a database and used as learning data for the AI ​​model.

[1833] This allows the system to not only propose optimal hairstyles based on the customer's requests and hair characteristics, but also provide personalized services that take into account the user's emotions and trend information.Furthermore, by incorporating feedback into the next proposal, customer satisfaction can be further increased.

[1834] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1835] Step 1:

[1836] User

[1837] The user launches the application and logs in. After logging in, a photo upload screen is displayed. The user selects a photo from the photo library or takes a new photo using the device's camera. The input is the user's photo data, and the output is the uploaded image data. Specifically, the user taps the smartphone app to launch it, and then enters their username and password for authentication. After authentication, the user selects or takes a photo and sends it to the server.

[1838] Step 2:

[1839] server

[1840] The server receives the uploaded image data and stores it in a database. It also analyzes it for the next step. The input is the uploaded image data, and the output is the image data stored in the database. Specifically, it receives an HTTP request and stores the image data in the database.

[1841] Step 3:

[1842] User

[1843] The user moves to the styling request input screen, enters information such as hair volume, texture, length, and style, and sends it to the server. The input is the request data entered by the user, and the output is the request data sent. Specifically, the user selects hair characteristics on the request input screen and taps the "Send" button.

[1844] Step 4:

[1845] server

[1846] The server stores the received request data in a database and analyzes it together with the image data. The input is the image data and the request data, and the output is the analyzed hair feature data. This analysis is performed using OpenCV and TensorFlow. Specifically, an image recognition algorithm is used to identify features such as hair length, volume, texture, and color.

[1847] Step 5:

[1848] server

[1849] The server uses an AI model (e.g., GAN) to generate the optimal hairstyle based on the analyzed hair features and desired data. The input is the analyzed hair feature data and desired data, and the output is the generated hairstyle data. Specifically, the server inputs a prompt statement into the AI ​​model to generate multiple hairstyles.

[1850] Step 6:

[1851] server

[1852] The server saves the generated hairstyle data in a database and sends it to the device. The input is the generated hairstyle data, and the output is the hairstyle suggestion data sent to the device. The specific operation is to save the results in a database and send them to the device as an HTTP response.

[1853] Step 7:

[1854] User

[1855] The user reviews the hairstyle suggestions generated on the device and enters feedback. The input is the proposed hairstyle data, and the output is the feedback data. Specifically, the user browses the suggested hairstyles on the app, selects them, and enters comments.

[1856] Step 8:

[1857] server

[1858] The server stores the received feedback data in a database and reflects it in the next proposal. The input is the feedback data, and the output is an updated database. Specifically, the server analyzes the feedback information and updates the database to use it in the next proposal.

[1859] Step 9:

[1860] server

[1861] The server uses an external API to collect the latest trend information. The input is the trend information obtained from the external API, and the output is the trend information stored in the database. Specifically, the server periodically sends an API request to obtain and store the latest trend information data.

[1862] Step 10:

[1863] server

[1864] The server performs a virtual coloring simulation for the generated hairstyle. The input is the generated hairstyle data and the user's skin tone data, and the output is multiple color variation data. Specifically, the server uses the coloring simulation model to generate different color variations of the hairstyle and save them in a database.

[1865] Step 11:

[1866] server

[1867] The server analyzes the user's facial expressions and voice to recognize emotions. The input is the user's facial expression and voice data, and the output is the recognized emotion data. Specifically, it calls the Emotion API, analyzes the user's facial expression and voice data, and identifies the emotion.

[1868] Step 12:

[1869] server

[1870] The server optimizes the suggestions based on the recognized emotion. The input is the recognized emotion data and the suggestion data, and the output is the suggestion data adjusted based on the emotion. Specifically, the server uses the AI ​​model to reevaluate the suggestions and reconfigure the optimal hairstyle based on the emotion.

[1871] (Application example 2)

[1872] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1873] Conventional hair salon application systems suggest hairstyles based on customer requests, but do not provide personalized suggestions based on the customer's emotions, making it difficult to maximize customer satisfaction. In addition to simply suggesting hairstyles, salons are also required to provide coloring simulations and trend information, but there is a lack of systems that provide these functions in an integrated manner. Furthermore, there is a need for a system that can incorporate real-time emotion recognition to make optimal suggestions based on the customer's emotions.

[1874] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1875] In this invention, the server includes a means for receiving image data of a customer, a means for receiving the customer's hairstyle requests, a means for analyzing the image data to extract the customer's hair characteristics, a means for generating an optimal hairstyle based on the customer's requests and hair characteristics, a means for proposing the generated hairstyle to the customer, and a means for analyzing the customer's facial expressions and voice to recognize emotions and adjust the proposed result. This makes it possible to propose an optimal hairstyle based on the customer's emotions in real time, in addition to their requests. It is also possible to perform coloring simulations and propose additional hairstyles based on the latest trend information, which can significantly improve overall customer satisfaction.

[1876] The "means for receiving customer image data" is a device that provides an interface for users to upload their photos to the system.

[1877] The "means for receiving customer requests regarding hairstyle" is a device that provides an interface through which the user can input information regarding the hairstyle and coloring they desire.

[1878] The "means for analyzing image data to extract characteristics of the customer's hair" is a device that includes an algorithm that analyzes uploaded image data and identifies characteristics such as hair length, texture, and color.

[1879] The "means for generating the optimal hairstyle based on the customer's requests and hair characteristics" is a device that integrates the user's requests with analyzed hair characteristics and generates the optimal hairstyle using an AI model.

[1880] The "means for proposing generated hairstyles to a customer" is a device that presents the hairstyles generated by the system to the user and provides an interface for receiving selections and feedback.

[1881] The "means for analyzing the customer's facial expressions and voice to recognize emotions and adjust the suggested results" refers to a device that includes an algorithm that analyzes the user's facial expressions and voice, recognizes emotions in real time, and adjusts the suggested hairstyle based on the results.

[1882] This invention relates to a system for improving customer experience in beauty salons. Specifically, the system receives customer image data and requests, analyzes them to suggest hairstyles and colorings, and recognizes emotions to adjust the suggestions, providing a more personalized service.

[1883] The system includes, among other things, the following main components:

[1884] 1. Means of receiving customer image data

[1885] It is mainly implemented in devices such as smartphones and tablets, and provides an interface for users to upload their own photos. For example, a smartphone application allows a user to take a selfie with their camera and upload it to the system. This photo data is then sent to a cloud server.

[1886] 2. A means of receiving customer requests regarding hairstyles

[1887] It provides an interface for users to input their desired hairstyle and coloring information. For example, users can input detailed requests such as hair length, texture, color, and style preferences into the app.

[1888] 3. A method for analyzing image data to extract the characteristics of a customer's hair

[1889] Using image recognition technology, the uploaded photo is analyzed to extract features such as hair length, texture, and color. This analysis can be performed using libraries such as TensorFlow and OpenCV. The analysis results are used in the next step.

[1890] 4. A means to generate optimal hairstyles based on customer requests and hair characteristics

[1891] Using a generative AI model, we generate the optimal hairstyle based on the user's request and analyzed hair characteristics. We use deep learning frameworks such as Keras and PyTorch to propose several hairstyle candidates.

[1892] 5. A means of proposing generated hairstyles to customers

[1893] The proposed hairstyles are presented to the user and an interface is provided for selection and feedback. For example, the generated hairstyles are displayed on the app, and the user can browse them and select the style they prefer.

[1894] 6. A method to recognize emotions by analyzing customer facial expressions and voice and adjust the proposal results

[1895] As users review the suggested hairstyles, the system analyzes their facial expressions and voice to recognize their emotions. This process is carried out using emotion recognition software such as EmotionEngine. Based on the recognized emotions, the system adjusts the suggestions to improve user satisfaction.

[1896] Specific examples

[1897] Let's take a specific example where customer C is looking for a new hairstyle at a hair salon.

[1898] 1. Upload a photo

[1899] Customer C opens the app, takes a photo of themselves, and uploads it.

[1900] 2. Input your request

[1901] Hair length: Medium

[1902] Hair Texture:Wave

[1903] Favorite color: Dark

[1904] 3. Data Analysis

[1905] It is analyzed as follows: Length: Medium, Texture: Wavy, Color: Black.

[1906] 4. Hairstyle Generation

[1907] Based on the analysis results, several new styles are generated.

[1908] 5. Emotion recognition

[1909] Recognize the happy expression on Customer C's face when he sees the proposed style.

[1910] 6. Final proposal result

[1911] The tailored suggestions are displayed to Customer C on her smartphone, and she chooses the style she likes best.

[1912] Prompt Sentence Examples

[1913] Image file: "path_to_your_image_file.jpg"

[1914] Customer Request:

[1915] Hair Length: Medium

[1916] Hair Texture: Wavy

[1917] Favorite color: Dark

[1918] Suggested style: "Curl"

[1919] Emotion recognition result: "Joy"

[1920] Make your final offer based on emotion.

[1921] This allows users to not only receive suggestions for the best hairstyle and coloring simulation based on their needs, but also receive more personalized suggestions based on their emotions, providing a highly satisfying experience.

[1922] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1923] Step 1:

[1924] Upload a photo

[1925] Subject: User

[1926] Description: A user launches the application, takes a photo of themselves, and uploads it to the system. The input is image data taken with the smartphone camera, and the output is data in an image file format sent to the cloud server. Specifically, the user presses the "Upload Photo" button, selects a photo, and uploads it.

[1927] Step 2:

[1928] Request input

[1929] Subject: User

[1930] Description: A user uses an application interface to input detailed information about their desired hairstyle and color. The input is the user's style preferences provided through text and options, and the output is data about those preferences sent to the server. Specific operations include selecting length, texture, color, etc. on the "Enter Hairstyle Preferences" screen and submitting it.

[1931] Step 3:

[1932] Data analysis

[1933] Subject: Server

[1934] Description: The server analyzes and processes the image data received and extracts the user's hair features. The input is the uploaded image data, and the output is the analysis results such as hair length, texture, and color. Specifically, it uses TensorFlow and OpenCV to detect hair areas from the image and extracts their features as numerical data.

[1935] Step 4:

[1936] Hairstyle generation

[1937] Subject: Server

[1938] Description: The server uses a generative AI model to generate the optimal hairstyle based on the analyzed hair features and the user's requests. The input is hair feature data and user request data, and the output is generated hairstyle candidates. Specifically, it applies a trained model using Keras or PyTorch to generate several optimal hairstyles.

[1939] Step 5:

[1940] Providing proposal results

[1941] Subject: Server

[1942] Description: The generated hairstyle is proposed to the user and made available for viewing. The input is the data of the generated hairstyle, and the output is the proposal information sent to the user's device. Specifically, the application generates images of the multiple generated hairstyles and displays them on the application screen.

[1943] Step 6:

[1944] emotion recognition

[1945] Subject: Server

[1946] Description: The server analyzes the user's facial expressions and voice to recognize emotions. The input is the user's facial expression and voice data, and the output is the recognized emotion data. Specifically, it uses the Emotion Engine to analyze data obtained from the camera and microphone and quantifies the user's emotions.

[1947] Step 7:

[1948] Adjustment of proposed results

[1949] Subject: Server

[1950] Description: Adjusts suggested hairstyles based on emotion recognition results. The input is the recognized emotion data and initial hairstyle suggestion data, and the output is the adjusted final suggestion data. Specific operations include modifying suggested hairstyles or making additional suggestions based on the recognized emotion.

[1951] Step 8:

[1952] Provision of final proposal results

[1953] Subject: Server

[1954] Description: Provides the user with a final hairstyle proposal after adjustments. The input is the adjusted proposal data, and the output is the final proposal information sent to the user's device. Specific operations include displaying the adjusted hairstyle proposal in the application so that the user can confirm it.

[1955] Step 9:

[1956] Collecting and storing feedback

[1957] Subject: User and Server

[1958] Description: The user reviews the suggested styles, selects the one they like, and provides feedback. This feedback is sent to the server and saved. The input is the hairstyle selected by the user and the feedback data, and the output is the feedback data saved on the server. Specifically, when the user presses the "Send Feedback" button, the information entered is sent to the server and used for the next suggestion.

[1959] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the control target 443 to output the result of the specific processing. The microphone 238 acquires voice indicating a user input regarding the result of the specific processing. The control unit 46A transmits voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[1960] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1961] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the robot 414.

[1962] The emotion identification model 59 as an emotion engine may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to an emotion map (see FIG. 9), which is a specific mapping. Similarly, the emotion identification model 59 may determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[1963] FIG. 9 is a diagram illustrating an emotion map 400 on which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. Emotions closer to the center of the concentric circles are more primitive. Emotions representing states and actions arising from a state of mind are arranged on the outer edges of the concentric circles. The concept of emotion includes both affect and mental states. Emotions generally generated from reactions occurring in the brain are arranged on the left side of the concentric circles. Emotions generally induced by situational judgment are arranged on the right side of the concentric circles. Emotions generally generated from reactions occurring in the brain and induced by situational judgment are arranged on the upper and lower sides of the concentric circles. Furthermore, the emotion of "pleasure" is arranged on the upper side of the concentric circles, and the emotion of "discomfort" is arranged on the lower side. In this way, in the emotion map 400, multiple emotions are mapped based on the structure by which emotions are generated, and emotions that tend to occur simultaneously are mapped close to each other.

[1964] These emotions are distributed in the 3 o'clock direction on emotion map 400, and typically fluctuate between relief and anxiety. In the right half of emotion map 400, situational awareness dominates over internal sensations, resulting in a sense of calm.

[1965] The inside of emotion map 400 represents what is going on in the mind, and the outside of emotion map 400 represents behavior, so the further you go outside emotion map 400, the more visible the emotions become (the more they are expressed in behavior).

[1966] Human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. Emotions can also be created for robots, automobiles, and motorcycles, based on various balances, such as posture and remaining battery life. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. An emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on Voice Emotion Recognition and Emotional Brain Physiological Signal Analysis Systems, Tokushima University, Doctoral Dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map lists emotions belonging to the "reaction" domain, where sensation is dominant. The right half of the emotion map lists emotions belonging to the "situation" domain, where situational awareness is dominant.

[1967] The emotion map defines two emotions that promote learning. One is a negative emotion on the situation side, around the middle of "repentance" or "reflection." In other words, this occurs when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is a positive emotion on the response side, around "desire." In other words, this occurs when the robot experiences positive feelings such as "I want more" or "I want to know more."

[1968] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​indicating each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple pieces of training data that are combinations of user input and emotion values ​​indicating each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions that are located close to each other have similar values, as in the emotion map 900 shown in FIG. 10. FIG. 10 shows an example in which multiple emotions, "relieved," "calm," and "reassuring," have similar emotion values.

[1969] The system according to the present disclosure has been described above mainly with respect to the functions of the data processing device 12, but the system according to the present disclosure is not necessarily implemented on a server. The system according to the present disclosure may be implemented as a general information processing system. The present disclosure may be implemented, for example, as a software program running on a personal computer or an application running on a smartphone, etc. The method according to the present disclosure may be provided to users in the form of SaaS (Software as a Service).

[1970] In the above embodiment, an example was given in which the specific processing is performed by one computer 22, but the technology of the present disclosure is not limited to this, and the specific processing may be distributed and performed by a plurality of computers including the computer 22. For example, the data generation model 58 may be provided in an external device of the data processing device 12, and data may be generated in the external device in accordance with input data.

[1971] In the above embodiment, an example in which the specific processing program 56 is stored in the storage 32 has been described, but the technology of the present disclosure is not limited to this. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-transitory storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-transitory storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes the specific processing in accordance with the specific processing program 56.

[1972] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[1973] It is not necessary to store all of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store all of the specific processing program 56 in the storage 32; only a portion of the specific processing program 56 may be stored.

[1974] The hardware resource for executing a specific process can be any of the following processors: An example of a processor is a CPU, which is a general-purpose processor that functions as a hardware resource for executing a specific process by executing software, i.e., a program. Another example of a processor is a dedicated electrical circuit, such as an FPGA (Field-Programmable Gate Array), a PLD (Programmable Logic Device), or an ASIC (Application Specific Integrated Circuit), which is a processor with a circuit configuration designed specifically for executing a specific process. Each processor has built-in or connected memory, and each processor uses the memory to execute the specific process.

[1975] The hardware resource that executes the specific processing may be configured with one of these various processors, or may be configured with a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Also, the hardware resource that executes the specific processing may be a single processor.

[1976] As an example of a system configured with a single processor, first, one processor is configured by combining one or more CPUs and software, and this processor functions as a hardware resource that executes a specific process. Second, there is a system that uses a processor that realizes the functions of an entire system including multiple hardware resources that execute a specific process on a single IC chip, as typified by SoC (System-on-a-chip). In this way, a specific process is realized using one or more of the above-mentioned various processors as hardware resources.

[1977] Furthermore, the hardware structure of these various processors can be, more specifically, an electric circuit that combines circuit elements such as semiconductor devices. The specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps may be deleted, new steps may be added, or the processing order may be rearranged, without departing from the spirit of the invention.

[1978] The above-described description and illustrations are a detailed explanation of the parts related to the technology of the present disclosure and are merely an example of the technology of the present disclosure. For example, the above description of the configuration, functions, actions, and effects is an explanation of an example of the configuration, functions, actions, and effects of the parts related to the technology of the present disclosure. Therefore, it goes without saying that unnecessary parts may be deleted, new elements may be added, or replacements may be made to the above-described description and illustrations within the scope of the gist of the technology of the present disclosure. Furthermore, to avoid confusion and facilitate understanding of the parts related to the technology of the present disclosure, the above-described description and illustrations omit explanations of common technical knowledge that do not require particular explanation to enable the implementation of the technology of the present disclosure.

[1979] All publications, patent applications, and technical standards mentioned in this specification are herein incorporated by reference to the same extent as if each individual publication, patent application, or technical standard was specifically and individually indicated to be incorporated by reference.

[1980] The following is further disclosed regarding the above embodiment.

[1981] (Claim 1)

[1982] means for receiving customer image data;

[1983] means for receiving customer requests regarding hairstyles;

[1984] means for analyzing image data to extract characteristics of the customer's hair;

[1985] A means for generating an optimal hairstyle based on the customer's requests and hair characteristics;

[1986] A means for proposing the generated hairstyle to a customer;

[1987] A system including:

[1988] (Claim 2)

[1989] 10. The system of claim 1, further comprising means for performing a virtual coloring simulation on the generated hairstyle.

[1990] (Claim 3)

[1991] 10. The system of claim 1, further comprising means for collecting current trend information and suggesting additional trend-based hairstyles to the customer.

[1992] "Example 1"

[1993] (Claim 1)

[1994] means for receiving user image data;

[1995] means for receiving a user's request regarding a hairstyle;

[1996] means for analyzing image data to extract features of the user's hair;

[1997] A generation AI model means for generating an optimal hairstyle based on a user's request and hair characteristics;

[1998] means for suggesting the generated hairstyle to a user;

[1999] A system including:

[2000] (Claim 2)

[2001] 10. The system of claim 1, further comprising means for performing a virtual coloring simulation on the generated hairstyle.

[2002] (Claim 3)

[2003] 10. The system of claim 1, further comprising means for collecting the latest trend information and suggesting additional hairstyles based on the trends to the user.

[2004] "Application Example 1"

[2005] (Claim 1)

[2006] means for receiving customer image data;

[2007] means for receiving customer requests regarding hairstyles;

[2008] means for analyzing image data to extract characteristics of the customer's hair;

[2009] A means for generating an optimal hairstyle based on the customer's requests and hair characteristics;

[2010] A means for proposing the generated hairstyle to a customer;

[2011] A means for displaying the generated hairstyle in real time using a wearable device such as smart glasses;

[2012] A system including:

[2013] (Claim 2)

[2014] 10. The system of claim 1, further comprising means for performing a virtual coloring simulation on the generated hairstyle.

[2015] (Claim 3)

[2016] 10. The system of claim 1, further comprising means for collecting current trend information and suggesting additional trend-based hairstyles to the customer.

[2017] "Example 2: Combining Emotion Engines"

[2018] (Claim 1)

[2019] means for receiving customer image data;

[2020] means for receiving customer requests regarding hairstyles;

[2021] means for analyzing image data to extract characteristics of the customer's hair;

[2022] A means for generating an optimal hairstyle based on the customer's requests and hair characteristics;

[2023] A means for proposing the generated hairstyle to a customer;

[2024] a means for recognizing user sentiment and optimizing suggestions based thereon;

[2025] A way to receive feedback from customers and reflect it in future proposals,

[2026] A system including:

[2027] (Claim 2)

[2028] 10. The system of claim 1, further comprising means for performing a virtual coloring simulation on the generated hairstyle.

[2029] (Claim 3)

[2030] 10. The system of claim 1, further comprising means for collecting current trend information and suggesting additional trend-based hairstyles to the customer.

[2031] "Application example 2 when combining emotion engines"

[2032] (Claim 1)

[2033] means for receiving customer image data;

[2034] means for receiving customer requests regarding hairstyles;

[2035] means for analyzing image data to extract characteristics of the customer's hair;

[2036] A means for generating an optimal hairstyle based on the customer's requests and hair characteristics;

[2037] A means for proposing the generated hairstyle to a customer;

[2038] A method to analyze customer facial expressions and voices to recognize emotions and adjust the proposal results.

[2039] A system including:

[2040] (Claim 2)

[2041] 10. The system of claim 1, further comprising means for performing a virtual coloring simulation on the generated hairstyle.

[2042] (Claim 3)

[2043] 10. The system of claim 1, further comprising means for collecting current trend information and suggesting additional trend-based hairstyles to the customer. [Explanation of symbols]

[2044] 10, 210...

Claims

1. means for receiving customer image data; means for receiving customer requests regarding hairstyles; means for analyzing image data to extract characteristics of the customer's hair; A means for generating an optimal hairstyle based on the customer's requests and hair characteristics; A means for proposing the generated hairstyle to a customer; A system including:

2. 10. The system of claim 1, further comprising means for performing a virtual coloring simulation on the generated hairstyle.

3. 10. The system of claim 1, further comprising means for collecting current trend information and suggesting additional hairstyles based on the trends to the customer.

Images