System

Abstract

A system is provided.SOLUTION: A system comprising: means for inputting a facial image and personal information of a user; means for analyzing the facial image and the personal information to extract feature data; means for generating a plurality of personalized style candidates based on the extracted feature data; means for presenting the generated style candidates to the user in real time; and means for notifying a hairdresser of a style result selected by the user.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] Traditionally, choosing a hairstyle at a hair salon often relied on magazines or abstract images, resulting in users frequently being dissatisfied with the results. It was difficult for users to find the style that best suited them, and communication with the hairdresser was often not smooth. Furthermore, detailed suggestions based on the user's skin condition or personal preferences were rarely made, making it difficult to provide a comprehensive beauty experience. [Means for solving the problem]

[0005] The present invention solves the above-mentioned problems by providing a system including: a means for inputting a user's facial image and personal information; a means for analyzing the facial image and personal information and extracting feature data; a means for generating a plurality of personalized style candidates based on the extracted feature data; a means for presenting the generated style candidates to the user in real time; and a means for notifying a hairdresser of the style results selected by the user. Furthermore, by adding a means for proposing styles and colors based on the user's skin condition and a means for the hairdresser to provide feedback on the results of the user's selection from the presented style candidates, the system realizes sophisticated style suggestions tailored to the user's individual needs and smooth communication with the hairdresser.

[0006] "User" refers to an individual who uses the salon's system to select their own hairstyle and color.

[0007] "Facial Image" refers to image data captured in real time of a user's face or a still image associated with the user.

[0008] "Personal information" refers to information about a user's individual characteristics, such as hair type, bone structure, skin condition, and preferred colors and styles.

[0009] "Analysis" refers to the process of processing a user's facial image and personal information to extract feature data.

[0010] "Feature data" refers to information obtained through analysis, such as the user's facial shape and hair type.

[0011] "Personalized style suggestions" refers to multiple hairstyle and color options suggested by a generative AI model based on the user's feature data.

[0012] "Generation" refers to the process of creating style suggestions, including appropriate hairstyles and colors, based on the user's feature data.

[0013] "Real-time" refers to a state in which the system provides results almost immediately after the user makes an input or selection.

[0014] "Presenting" refers to showing the generated style candidates to the user.

[0015] "Selection" refers to the act of the user selecting one or more style candidates that suit their preferences from the multiple style candidates presented.

[0016] "Notification" refers to the process of informing the hairdresser of the style results selected by the user.

[0017] A "hairdresser" is a professional who works in a beauty salon, providing hairstyles and advice to customers.

[0018] "Feedback" refers to the act of a hairdresser providing feedback or advice about the style selected by the user.

[0019] "Skin condition" is information that indicates the user's skin condition, and style and color suggestions are made based on this.

[0020] "System" refers to an overall structure and collection of devices for carrying out a series of processes and functions in the present invention. [Brief explanation of the drawings]

[0021] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 3 is a sequence diagram showing a processing flow of the data processing system according to the first embodiment. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0022] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0023] First, the terms used in the following description will be explained.

[0024] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0025] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0026] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0027] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0028] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0029] [First embodiment]

[0030] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0031] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0032] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0033] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0034] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0035] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0036] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0037] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0038] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0039] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0040] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0041] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0042] The system of the present invention provides personalized hairstyle suggestions using AI technology to improve the user experience at a beauty salon. This system operates in cooperation with multiple means described below.

[0043] 1. Enter and submit user information

[0044] Terminal

[0045] Users use dedicated terminals in beauty salons to input their individual personal information and real-time facial images. The user captures a facial image using the terminal's camera and inputs information about hair type (e.g., straight or curly), bone structure, and preferred hairstyle and color. All this information is encrypted and sent to a server for further processing.

[0046] 2. Analysis and style candidate generation

[0047] server

[0048] The server receives the user's facial image and personal information sent from the device. Based on this data, an AI model is used to extract characteristic data such as facial shape, hair type, and skin condition, and based on that, optimal hairstyle and color candidates are generated. Multiple style candidates are generated and superimposed on the user's facial image.

[0049] 3. Presentation and selection of style candidates

[0050] Terminal

[0051] The generated style candidates are sent from the server to the device, which then presents the received multiple style candidates to the user in real time, allowing the user to check how the style will look on their actual appearance.

[0052] user

[0053] Users can choose the style they like best from a selection of suggested styles, and in some cases can refer to detailed information and comments to help them choose the appropriate style.

[0054] 4. Notification of results and feedback

[0055] Terminal

[0056] The selection results are sent to the hairdresser in real time, who then reviews the style selected by the user and begins preparations for the treatment.

[0057] hairdresser

[0058] Based on the user's chosen style, the stylist will hold a detailed consultation with the user before performing any necessary treatments. The stylist will provide feedback on the user's chosen style and make any final adjustments.

[0059] Specific examples

[0060] As a specific example, suppose User A visits a hair salon, captures an image of their face on their device, and enters their hair type as "straight" and their favorite color as "brown." The device sends this information to the server, which uses an AI model to generate multiple brown hairstyles. These styles are sent back to the device superimposed on User A's face image, allowing User A to view them in real time. User A then selects their favorite style, and the result is notified to the hairdresser. After reviewing and consulting with User A, the hairdresser begins treatment based on the selected style.

[0061] In this way, the system enhances communication between users and hairdressers, providing optimal styles tailored to individual needs and improving the salon experience.

[0062] The processing flow will be explained below.

[0063] Step 1:

[0064] The user operates the device, captures a facial image using the camera, and inputs personal information such as hair type, preferred color, and hairstyle.

[0065] Step 2:

[0066] The device sends the captured facial image and input personal information to a server, where the data is encrypted to protect privacy.

[0067] Step 3:

[0068] The server analyzes the received facial image data and personal information, and extracts characteristic data such as the user's facial shape and hair type using an image analysis algorithm.

[0069] Step 4:

[0070] The server inputs the extracted feature data into an AI model to generate multiple personalized hairstyle and color suggestions based on the user's specific information.

[0071] Step 5:

[0072] The server then sends the generated style candidate images to the device in real time, which then displays an image of the user's face with the style applied.

[0073] Step 6:

[0074] The device then presents the received style candidate images to the user, who can then view the style applied to their own face on the device screen.

[0075] Step 7:

[0076] Users can select the style they like best from the suggested styles, and can also view detailed information and comments about the selected style.

[0077] Step 8:

[0078] The device sends the user's selected style results to the server in real time.

[0079] Step 9:

[0080] The server processes the selection results and sends a notification to the hairdresser's device, where the hairdresser can see the user's selection results in real time.

[0081] Step 10:

[0082] After reviewing the user's chosen style, the hairstylist will hold a detailed consultation with the user and provide feedback as needed.

[0083] Step 11:

[0084] Based on the final feedback, the hairdresser will begin the actual treatment and create a hairstyle that meets the customer's wishes.

[0085] Example 1

[0086] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0087] Currently, hair salons have difficulty proposing hairstyles that effectively reflect the individual preferences and characteristics of their customers. Furthermore, there is a lack of means for customers to quickly communicate their chosen style to the hairdresser and incorporate their feedback. As a result, achieving the hairstyle the customer desires often requires time and effort.

[0088] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0089] In this invention, the server includes means for inputting a user's facial image and personal information, means for encrypting the facial image and personal information and transmitting them to the server, means for decrypting the encrypted data and extracting feature data in the server, means for generating a plurality of personalized style candidates using a generative AI model based on the extracted feature data, means for presenting the generated style candidates to the user in real time, and means for notifying the provider of the style results selected by the user. This enables the server to quickly propose an optimal hairstyle according to the user's individual preferences and facilitates smooth communication with the hairdresser.

[0090] A "user's facial image" is video data that includes the facial features of an individual user.

[0091] "Personal Information" refers to information about a user, such as their hair type, bone structure, and preferred styles and colors.

[0092] "Encryption" is the process of using a cryptographic standard (e.g., AES-256) to transmit data securely and converting the content into a format that is unreadable to third parties.

[0093] "Feature data" refers to user-specific data points and attributes extracted based on the user's facial image and personal information.

[0094] A "generative AI model" is a model that uses artificial intelligence technology to generate optimal hairstyle candidates from input data.

[0095] "Style suggestions" are multiple hairstyle options generated by the AI ​​model to suggest to the user.

[0096] "Real-time" refers to processing within a timeframe that provides an immediate response to user interaction.

[0097] A "provider" is a professional who provides a service to a user, such as a hairdresser.

[0098] "Feedback" is any evaluation, opinion, or advice from the provider regarding the style selected by the user.

[0099] This invention is a system for providing personalized hairstyle suggestions that reflect the individual preferences and characteristics of users at beauty salons. The system includes a series of techniques for encrypting data entered by users using a dedicated terminal, analyzing it on a server, and generating optimal style suggestions.

[0100] Hardware and software used

[0101] The server uses NVIDIA GPUs (e.g., NVIDIA RTX 3080) and TensorFlow to run AI models, and the OpenSSL library is used for data encryption and decryption. The device is equipped with a camera, which is used to capture the user's facial image.

[0102] System Operation Overview

[0103] Terminal

[0104] Users use a dedicated terminal at the salon to input their facial image and personal information (e.g., hair type, bone structure, preferred hairstyle and color). Specifically, users capture a facial image using the terminal's camera and input various personal information on the screen. All of this information is encrypted using AES-256 to ensure security. The encrypted data is then sent to a server via the Internet.

[0105] server

[0106] The server receives the encrypted data sent from the device and decrypts it using AES-256. The decrypted data includes the user's facial image and personal information. The server then runs an AI model using TensorFlow to extract the user's feature data from the facial image and personal information. Based on this feature data, the server generates multiple style candidates, including optimal hairstyles and colors. The generated style candidates are then overlaid on the user's facial image.

[0107] Terminal

[0108] The generated style candidates are sent from the server to the device. The device then presents these style candidates to the user in real time, allowing the user to see how they will look when applied to their own face. The user can then select the style candidate they like best from the multiple style candidates presented. If necessary, they can refer to detailed information and comments.

[0109] user

[0110] The user compares multiple style options displayed in real time and selects the hairstyle that best suits them. The result of this selection is then notified to the provider (e.g., hairdresser) in real time.

[0111] provider

[0112] The provider will review the user's selections on the device and begin preparing the necessary treatment. During a detailed consultation with the user, the provider will provide feedback based on the style selected and make any final adjustments.

[0113] Specific examples

[0114] For example, let's say User A visits a hair salon. First, User A uses a device to capture an image of their face, and enters their hair type as "straight" and their preferred color as "brown." This information is encrypted on the device and sent to the server. The server decrypts the encrypted data and uses an AI model to generate multiple brown hairstyle candidates. These generated style candidates are superimposed on User A's face image and sent back to the device. User A compares the displayed multiple style candidates and selects the one they like best. The selection result is notified to the salon provider, who then begins preparations for the treatment and, after a detailed consultation with User A, adjusts the style to the final version.

[0115] Prompt Sentence Examples

[0116] "Run an AI model to suggest the best hairstyle options based on the user's facial image and personal information."

[0117] This system quickly suggests the best hairstyle based on the user's individual preferences and enables smooth communication with the hairdresser, which is expected to improve the user experience at the salon and increase satisfaction.

[0118] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0119] Step 1:

[0120] The user inputs his / her facial image and personal information using a dedicated terminal at the beauty salon.

[0121] Specifically, users use the device's camera to capture a facial image and enter personal information such as hair type, bone structure, and preferred hairstyle and color.

[0122] Input: User's face image and personal information.

[0123] Output: The encrypted data.

[0124] Step 2:

[0125] The device encrypts the entered facial image and personal information using AES-256.

[0126] Specifically, the terminal encrypts input data and executes processes to maintain security.

[0127] Input: User's face image and personal information.

[0128] Output: The encrypted data.

[0129] Step 3:

[0130] The encrypted data is sent to a server over the Internet.

[0131] Specifically, the terminal executes a process of transmitting encrypted data to a server via a network.

[0132] Input: Encrypted data.

[0133] Output: The data sent to the server.

[0134] Step 4:

[0135] The server decrypts the received encrypted data using AES-256.

[0136] Specifically, the server runs a process to decrypt the encrypted data and retrieve the original facial image and personal information.

[0137] Input: Encrypted data.

[0138] Output: Decoded face image and personal information.

[0139] Step 5:

[0140] The server extracts feature data based on the decoded face image and personal information.

[0141] Specifically, the server runs an AI model to extract feature data from the user's facial image and personal information.

[0142] Input: Decoded face image and personal information.

[0143] Output: Extracted feature data.

[0144] Step 6:

[0145] The server generates multiple personalized style candidates using a generative AI model based on the extracted feature data.

[0146] Specifically, the server uses an AI model to generate multiple hairstyle options that are overlaid on the user's face image.

[0147] Input: Extracted feature data.

[0148] Output: Multiple style candidates.

[0149] Step 7:

[0150] The server sends the generated style candidates to the terminal in real time.

[0151] Specifically, a process is executed to transmit the generated style candidate data to the terminal via the Internet.

[0152] Input: Multiple style candidates.

[0153] Output: Data sent to the terminal.

[0154] Step 8:

[0155] The device presents the received style suggestions to the user in real time.

[0156] Specifically, the device executes a process to display style candidates superimposed on the user's facial image.

[0157] Input: Style suggestions received from the server.

[0158] Output: Multiple style suggestions displayed to the user.

[0159] Step 9:

[0160] The user selects the style they like best from the multiple style options presented.

[0161] Specifically, the user compares style candidates on the device screen and makes a selection.

[0162] Input: Several style suggestions displayed on the device.

[0163] Output: User selected style.

[0164] Step 10:

[0165] The device notifies the provider of the user's selected style results in real time.

[0166] Specifically, the terminal executes a process of transmitting the selection results to the provider in real time via the network.

[0167] Input: The style selected by the user.

[0168] Output: The selection results communicated to the provider.

[0169] Step 11:

[0170] The provider will review the user's selections and begin preparing the necessary treatment.

[0171] Specifically, the provider checks the selection results on the terminal and plans the treatment in consultation with the user.

[0172] Input: The selection results notified to the provider.

[0173] Output: Preparing and planning the procedure.

[0174] Step 12:

[0175] The provider will hold a detailed consultation with the user and provide feedback based on the style selected.

[0176] Specifically, the provider will actually have a conversation with the user and make final style adjustments.

[0177] Input: The selection results notified to the provider.

[0178] Output: Feedback and final adjustments provided to the user.

[0179] (Application example 1)

[0180] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0181] In modern society, the expectations and needs of users of beauty services are becoming more diverse. While remote home beauty treatments and virtual salon experiences are on the rise, choosing the hairstyle and color that best suits them can be a challenging task. Furthermore, the time and expense required to actually try on a selected style can reduce user satisfaction. Given this background, there is a demand for a system that allows users to try on various hairstyles in real time from the comfort of their own home and select the style that best suits them.

[0182] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0183] In this invention, the server includes a device for inputting a user's facial image and personal information, a device for analyzing the facial image and personal information and extracting feature data, a device for generating a plurality of personalized style candidates based on the extracted feature data, a device for presenting the generated style candidates to the user in real time, a device for notifying the technician of the style selected by the user, and a device for transmitting the selected style, presenting the style in real time, and generating a prompt message for accepting the selection. This allows the user to try out various hairstyles in real time from the comfort of their own home, select the best style, and immediately notify the technician.

[0184] "Device for inputting a user's facial image and personal information" refers to a terminal or application that allows a user to input their own facial photo and personal information (e.g., hair type, preferred color, etc.).

[0185] "Device that analyzes facial images and personal information and extracts feature data" refers to a server or system that processes input facial images and personal information and uses AI technology to analyze feature data such as facial shape and hair type.

[0186] A "device for generating multiple personalized style candidates" is a system that includes an algorithm or AI model that automatically generates optimal style and hair color candidates for a user based on extracted feature data.

[0187] The "device that presents generated style candidates to the user in real time" refers to a terminal or application that displays a plurality of generated style candidates to the user in real time for confirmation.

[0188] The "device for notifying a technician of the style results selected by a user" refers to a system for notifying a technician (e.g., a hairdresser or hair stylist) of the style information selected by a user in real time.

[0189] "A device that transmits selected styles, presents styles in real time, and generates prompts to accept selections" refers to a device that transmits style information and provides an interface for selection using prompts that are generated based on user selections.

[0190] The system that realizes this application example is composed of a device that inputs a user's facial image and personal information, a server that analyzes the facial image and personal information and extracts feature data, a server that generates personalized style candidates based on the extracted feature data, a terminal that presents the generated style candidates to the user in real time, a device that notifies the engineer of the selected style result, and a device that transmits the selected style and generates a prompt sentence.

[0191] Specific operation of the system

[0192] The operation of this system is as follows.

[0193] 1. Enter and submit user information

[0194] Users use a dedicated device or smartphone app to capture an image of their face and enter personal information such as hair type and preferred color. The data is encrypted and then sent to a server. Logitec webcams and other cameras are used as camera devices, and the Python OpenCV library is used for image processing.

[0195] 2. Analysis and style candidate generation

[0196] The submitted facial image and personal information are received by a server and analyzed using an AI model. The server extracts feature data such as facial shape and hair type, and generates multiple hairstyle candidates based on that data. This AI model is built using deep learning frameworks such as TensorFlow and Keras.

[0197] 3. Presentation and selection of style candidates

[0198] The generated hairstyle candidates are sent from the server to the user's device. The user can view the hairstyles in real time and select the style they like best. This process uses OpenCV image display technology. The selected style is then sent back to the server.

[0199] 4. Notification of results and feedback

[0200] The selection results are notified to the technician in real time, and the technician can use that information to prepare for the procedure. The technician can provide feedback to the user and make final adjustments. A prompt generation device is then used to generate a prompt based on the user's selection and send it to the technician.

[0201] Examples of prompt statements

[0202] A specific example of a prompt is as follows:

[0203] Based on the user's face image and the following information, please provide five suitable hairstyle options.

[0204] Hair Type: Straight

[0205] Favorite color: Brown

[0206] Facial images are provided in the following formats:

[0207] <Image data>

[0208] In this way, users can virtually try out different styles from the comfort of their own home, with the results being communicated to the technician in real time, which is expected to provide a smooth and efficient beauty experience.

[0209] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0210] Step 1:

[0211] Users use a dedicated device or smartphone app to capture an image of their face and enter personal information such as hair type and preferred color. The facial image is captured using a camera device (e.g., a Logitec webcam). The personal information is collected in text format based on the user's input. This data is encrypted and then sent to the server.

[0212] Input: Personal information such as the user's face image, hair type, favorite color, etc.

[0213] Output: Encrypted face image and personal information are sent to the server

[0214] Step 2:

[0215] The server analyzes the received facial image and personal information and extracts feature data from them. Specifically, it uses image analysis technology (e.g., OpenCV) to detect facial shape, hair type, skin condition, etc. It also analyzes the input personal information to extract information about preferred styles.

[0216] Input: Encrypted face image and personal information

[0217] Output: Feature data such as face shape, hair texture, skin condition, etc.

[0218] Step 3:

[0219] The server generates personalized hairstyle candidates based on the extracted feature data. To do this, it uses a generative AI model (e.g., a deep learning model using TensorFlow or Keras). This model generates multiple hairstyles that match the user's features.

[0220] Input: feature data

[0221] Output: Multiple personalized hairstyle suggestions

[0222] Step 4:

[0223] The server sends the generated hairstyle candidates to the user's device, which then presents them to the user in real time. Each proposed style is combined with the facial image, and the user can view the style displayed through the interface. This process uses image display technology (e.g., OpenCV).

[0224] Input: Hairstyle candidate

[0225] Output: Hairstyle suggestions presented in real time

[0226] Step 5:

[0227] The user selects the hairstyle they like best from the multiple hairstyle options presented, and the selection is sent back to the server from the device.

[0228] Input: User selection

[0229] Output: The selection results are sent to the server

[0230] Step 6:

[0231] The server notifies the technician of the user's selection. The technician uses this information to prepare for the treatment. Furthermore, a prompt message based on the user's selection is sent to the technician. This allows the technician to respond quickly to the user's request.

[0232] Input: User selection

[0233] Output: Generates and sends notifications and prompts to technicians

[0234] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0235] The system of the present invention provides personalized hairstyle suggestions by combining AI technology and an emotion engine to improve the user experience in a beauty salon. This system operates in cooperation with multiple means described below.

[0236] 1. Enter and submit user information

[0237] Terminal

[0238] The user uses a dedicated terminal at the salon to input their personal information and real-time facial image. The user captures the facial image using the terminal's camera, and the user's hair type (e.g., straight or curly), bone structure, preferred hairstyle and color, and the emotion engine reads the user's emotions from their facial expressions. All of this information is encrypted and sent to the server.

[0239] 2. Analysis and style candidate generation

[0240] server

[0241] The server receives the user's facial image, personal information, and emotional data sent from the device. Based on this data, an AI model is used to extract feature data such as facial shape, hair type, skin condition, and emotional data obtained by an emotion engine, and based on this, optimal hairstyle and color candidates are generated. Multiple style candidates are generated and superimposed on the user's facial image.

[0242] 3. Presentation and selection of style candidates

[0243] Terminal

[0244] The generated style candidates are sent from the server to the device, which then presents the received multiple style candidates to the user in real time, allowing the user to check the style applied to their own face.

[0245] user

[0246] Users can choose the style they like best from a selection of suggested styles. In some cases, they can refer to detailed information and comments to select the appropriate style. Additionally, the system analyzes users' emotions in real time and reflects this information in the style selection.

[0247] 4. Notification of results and feedback

[0248] Terminal

[0249] The selection results are sent to the hairdresser in real time, who then reviews the style selected by the user and begins preparations for the treatment.

[0250] hairdresser

[0251] Based on the user's selections and emotional data, the hairdresser will hold a detailed consultation with the user and provide feedback as needed.

[0252] Specific examples

[0253] As a specific example, suppose User B visits a hair salon, captures an image of their face on their device, and enters their hair type as "curly" and their favorite color as "blonde." The device sends this information, along with the user's emotional data, to the server. The server uses an AI model and an emotion engine to generate multiple blonde hairstyles, which are then superimposed on User B's facial image and sent back to the device. User B reviews the style options presented in real time and selects the most suitable one. The emotional data is also reflected, and the user's selected style is notified to the hairdresser. The hairdresser reviews the selection and emotional data, prepares for the treatment, consults with User B, and makes any final adjustments before beginning the treatment.

[0254] In this way, by incorporating the user's emotional data, this system can provide more personalized style suggestions, improve user satisfaction, and enhance communication with the hairdresser.

[0255] The processing flow will be explained below.

[0256] Step 1:

[0257] The user operates the device, captures a facial image using the camera, and inputs personal information such as hair type, preferred color, and hairstyle. The emotion engine then analyzes the user's facial expressions and collects emotional data.

[0258] Step 2:

[0259] The device transmits the captured facial image, input personal information, and acquired emotion data to a server, where the data is encrypted to protect privacy.

[0260] Step 3:

[0261] The server analyzes the received facial image data, personal information, and emotional data, and extracts feature data based on the user's facial shape, hair type, skin condition, and emotional data using image analysis and emotional analysis algorithms.

[0262] Step 4:

[0263] The server inputs the extracted feature data and emotion data into an AI model to generate multiple personalized hairstyle and color suggestions based on the user's unique information and current emotion. Emotion data is given special weight when generating style suggestions.

[0264] Step 5:

[0265] The server sends the generated style candidate images to the device in real time, and the device displays the received style candidates superimposed on the user's facial image, allowing the user to check them in real time.

[0266] Step 6:

[0267] The device presents style candidates to the user, who then sees the style applied to their face. The proposed style takes into account the user's current emotional state, making it highly appropriate.

[0268] Step 7:

[0269] The user selects the style they like best from the multiple style options presented, and the selection reflects the user's emotional state, increasing satisfaction.

[0270] Step 8:

[0271] The device transmits the user's selected style results, including emotional data, to the server in real time.

[0272] Step 9:

[0273] The server processes the selection results and emotion data and sends a notification to the hairdresser's device, where the hairdresser can check the selection results and emotion data in real time.

[0274] Step 10:

[0275] The hairstylist will then hold an in-depth consultation with the user based on their selection and emotional data, providing feedback that takes their emotions into account and suggesting tweaks to the style if necessary.

[0276] Step 11:

[0277] Based on the final feedback, the hairdresser will begin the actual treatment and achieve the hairstyle that matches the user's wishes. Emotional data is taken into consideration throughout the treatment, improving the user experience.

[0278] In this way, by adding emotional data, this system can provide more personalized style suggestions, improve user satisfaction, and enhance communication with hairdressers.

[0279] Example 2

[0280] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0281] To improve customer satisfaction at hair salons, personalized style suggestions that take into account the individual characteristics and emotions of each user are required. However, conventional systems only consider basic information such as the user's facial image, hair type, and preferred style, and are unable to suggest styles that reflect detailed personal data such as the user's emotions and skin condition. This results in issues such as not being able to meet user expectations and not being able to communicate effectively with the hairdresser.

[0282] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0283] In this invention, the server includes a means for acquiring the user's emotional data and reflecting it in the creation of a style, a means for proposing a style and color based on the user's skin condition, and a means for the hairdresser to provide feedback on the result of the user's selection from the presented style candidates. This makes it possible to propose a style based on detailed personal data such as the user's facial image, hair type, and preferred style, as well as their emotions and skin condition.

[0284] A "facial image" is a digital image that captures the visual features of a user's face.

[0285] "Personal information" refers to information about the user, such as hair type, preferred hairstyle and color.

[0286] "Feature data" refers to characteristic data such as facial shape, hair type, and skin condition extracted from the user's facial image and personal information.

[0287] "Style candidates" are multiple hairstyle and color suggestions generated based on feature data, and are presented as a composite image superimposed on the user's facial image.

[0288] An "emotion engine" is software or hardware that reads emotions from a user's facial expressions and analyzes them as data.

[0289] "Real-time" refers to a state in which data and processing occur with almost no delay, allowing users and systems to respond immediately.

[0290] "Feedback" refers to any comments or information provided by a Hairdresser regarding improvements or evaluation of a User's selection.

[0291] "Suggestions" are specific style and color options and ideas to present to users.

[0292] The system of the present invention provides personalized hairstyle suggestions by combining AI technology and an emotion engine to improve the user experience in a beauty salon. This system utilizes the user's facial image, personal information, and emotion data, and includes a series of processes to generate and present optimal style suggestions in real time.

[0293] First, the user uses a dedicated terminal installed in the salon to input their individual personal information and a real-time facial image. The terminal is equipped with a camera, and the user presses the "take a photo" button to capture a facial image. Along with this facial image, the user also inputs their hair type, preferred hairstyle and color. In addition, the emotion engine installed in the terminal reads emotions from the user's facial expressions, and this information is also collected. This data is encrypted and sent to the server.

[0294] The server analyzes the received facial image, personal information, and emotional data. First, it uses a facial recognition algorithm to extract the shape and features of the face. It then analyzes the acquired emotional data using an emotion engine to recognize specific emotions. Based on this analyzed data, the server uses an AI model to generate optimal hairstyle and color candidates. Multiple style candidates are generated and superimposed on the user's facial image.

[0295] The generated style candidates are then sent from the server to the device. The device then presents the received style candidates to the user in real time. The user can then select the style they like best from the multiple style candidates presented. When making a selection, the user can refer to detailed information and comments for each style. In addition, the user's emotions are analyzed in real time, and this information is reflected in the style selection.

[0296] Once the user selects the best style, the results are communicated to the hairdresser in real time. The hairdresser then conducts a detailed consultation with the user based on the selected style and emotional data, providing feedback as needed. The hairdresser then begins preparations for the treatment, making any final adjustments before styling the user's hair.

[0297] As a concrete example, suppose User B visits a hair salon, captures a facial image on a device, and inputs "hair type: curly" and "preferred color: blonde." This data and emotional data are sent to the server. The server uses an AI model and an emotion engine to generate a blonde style for User B and overlays it on the user's facial image. The generated style candidates are displayed on the device, and User B selects the most suitable style. The selection result, reflecting User B's emotions, is notified to the hairdresser, who makes preparations based on that information and begins the treatment after a final consultation.

[0298] An example prompt is:

[0299] "I'm a woman in her 20s with a cheerful face and curly hair. My favorite color is blonde. Please generate the best hairstyle for her."

[0300] By incorporating user emotional data, the system can provide personalized style suggestions, improve user satisfaction, and enhance communication with hairdressers.

[0301] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0302] Step 1: Enter your user information

[0303] Terminal

[0304] The user uses a dedicated terminal at the salon and first enters their personal information. The terminal is equipped with a camera, and the user captures a facial image by facing the camera and pressing the "shoot" button. Specifically, the facial image (input) is obtained and the data is temporarily saved (output). The user also inputs their hair type (e.g. straight hair, curly hair), preferred hairstyle and color (input) through the UI. Along with this information, the emotion engine analyzes facial expressions and obtains emotional data (output).

[0305] Step 2: Submit user information

[0306] Terminal

[0307] The captured facial image, input hair type, preferred hairstyle and color, and acquired emotion data are all encrypted. The encrypted data packet is sent to the server. The input is the entire user information, and the output is the encrypted data packet.

[0308] Step 3: Receiving and analyzing user information

[0309] server

[0310] The server receives encrypted data packets from the device (input). The received data is decrypted and separated into facial images, personal information, and emotional data (output). The facial images are then analyzed using a facial recognition algorithm to extract facial shapes and features, generating feature data (output). The emotion engine then analyzes the acquired emotional data to identify specific emotions (e.g., joy, surprise, sadness).

[0311] Step 4: Generate style candidates

[0312] server

[0313] The server operates a generative AI model based on the analyzed feature data and emotional data (input). This AI model combines the user's face shape, hair type, personal information, and emotional data to generate multiple candidates for optimal hairstyles and colors (output). It also creates a composite image in which these styles are superimposed on the user's face image. Style candidate data including this composite image is generated.

[0314] Step 5: Submit and present your style suggestions

[0315] server

[0316] The generated style candidates and the composite image are sent to the terminal (the input is the style candidate data, and the output is the sent data). This is also encrypted and transmitted securely.

[0317] Terminal

[0318] The device decodes the received style candidate data (input) and presents multiple style candidates and their detailed information to the user in real time (output). The user can check each style through the UI. For example, pressing the "Next" button displays the next style candidate.

[0319] Step 6: User Style Selection

[0320] user

[0321] The user selects the style they like best from the multiple style candidates presented (input: user selection). By pressing the "Select" button on the UI, that style is selected (output). If necessary, detailed information and comments for each style can be viewed to help with the final decision. Additionally, the emotion engine analyzes the user's facial expressions while selecting and reflects them in the selection results.

[0322] Step 7: Notification of results and feedback

[0323] Terminal

[0324] The server notifies the user of the style they have selected in real time (the input is the selection result, and the output is the notification data). This information is then forwarded to the hairdresser's device.

[0325] hairdresser

[0326] The hairdresser confirms the user's selection and emotional data (input) and begins preparations for the treatment. He / she then consults with the user (output) and provides necessary feedback, such as explaining the feasibility of the selected style and the necessary treatments. After making any final adjustments, the treatment begins.

[0327] Through this series of processes, personalized style suggestions and high-quality treatments are realized based on the user's emotions and personal information.

[0328] (Application example 2)

[0329] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0330] Hair salons need to propose optimal hairstyles based on the individual feelings and preferences of their customers, thereby increasing customer satisfaction. However, conventional systems have the problem that customers cannot confirm which style suits them until they actually visit the salon. Furthermore, there is no way for users to try the style at home before the treatment, and due to a lack of communication with the hairdresser, the desired style may not be fully reflected.

[0331] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0332] In this invention, the server includes: means for inputting a user's facial image and personal information; means for analyzing the facial image and personal information and extracting feature data; means for generating multiple personalized style candidates based on the extracted feature data; means for presenting the generated style candidates to the user in real time; means for notifying the hairdresser of the user's selected style; means for the user to use the virtual beauty salon app at home; means for capturing a facial image and inputting a desired hairstyle using a smartphone; and means for generating a hairstyle in real time using an AI model and an emotion engine through the virtual beauty salon app. This allows the user to try out multiple hairstyles at home and select the best style before visiting the salon. As a result, user satisfaction is improved and communication with the hairdresser is smoother.

[0333] "User" refers to an individual who uses the services of a beauty salon.

[0334] "Facial image" means a photograph or image data of a user's face.

[0335] "Personal Information" refers to your hair type, preferred hairstyles and colors, and other personal information.

[0336] "Feature data" refers to data such as facial shape, hair type, and skin condition analyzed from the input facial image and personal information.

[0337] "Style suggestions" refers to multiple hairstyles suggested to the user.

[0338] "Real-time" means immediate response or processing without delay.

[0339] "Practitioner" refers to a professional who performs hairstyles in a beauty salon, i.e., a hairdresser.

[0340] A "virtual beauty salon app" refers to application software that allows users to try out hairstyles at home using their smartphones.

[0341] "Smartphone" means a portable communication device that has Internet connectivity and can be used by installing applications.

[0342] An "AI model" refers to artificial intelligence that uses technologies such as machine learning and deep learning to analyze a user's characteristic data and generate optimal style candidates.

[0343] "Emotion engine" refers to technology that reads emotions from a user's facial expressions and uses those emotions as data for analysis.

[0344] The system of the present invention provides personalized hairstyle suggestions that combine AI technology and an emotion engine to improve the user experience in beauty salons. This system includes a means for inputting a user's facial image and personal information, a means for analyzing the facial image and personal information and extracting feature data, a means for generating multiple personalized style candidates based on the extracted feature data, a means for presenting the generated style candidates to the user in real time, and a means for notifying the hairdresser of the style results selected by the user.

[0345] 1. Enter and submit user information

[0346] Terminal

[0347] Users use their smartphone to capture a photo of their face at home or at a dedicated terminal in a beauty salon, and input their hair type, preferred hairstyle, and color. The facial image is then captured using the smartphone's camera or the dedicated terminal's camera. This data is encrypted as needed and sent to a server.

[0348] 2. Analysis and style candidate generation

[0349] server

[0350] After receiving the user's personal information and facial image, the server uses an AI model to analyze facial shape, hair type, skin condition, and even emotional data obtained from an emotion engine to extract feature data. Multiple style candidates are then generated based on the extracted feature data. The generated style candidates are then overlaid on the user's facial image to present a realistic image.

[0351] 3. Presentation and selection of style candidates

[0352] Terminal

[0353] The generated style candidates are sent from the server to the device and are presented to the user in real time on a smartphone or dedicated device. The user can then review multiple style candidates applied to their face and select the one that best suits them.

[0354] user

[0355] Users can select their favorite style from the suggested styles, and the system will also incorporate suggestions based on emotional data to optimize the selection.

[0356] 4. Notification of results and feedback

[0357] Terminal

[0358] The results of the selected style are notified to the practitioner in real time, who then confirms the style selected by the user and begins preparations for the treatment.

[0359] Practitioner

[0360] The practitioner will conduct a detailed consultation based on the user's selections, provide feedback as needed, and make any final adjustments before beginning the treatment.

[0361] Hardware and software used

[0362] In this invention, in addition to the camera of a smartphone or dedicated device, a server is used to run the AI ​​model, and existing AI services such as Azure Face API, Google Cloud Vision, and Amazon Rekognition are used for the analysis.

[0363] Specific examples

[0364] For example, when User A uses this system at home, they capture a facial image using their smartphone and enter their hair type as "curly" and color as "red." The smartphone encrypts this information and sends it to the server. The server-side AI model analyzes this data, generates multiple personalized style options for User A, and sends the results to the smartphone. Once User A selects the optimal style, the results are notified to the hairdresser in real time.

[0365] Prompt Sentence Examples

[0366] "Generate the best hairstyle candidates for the user based on their face image and preferred hairstyle information. The hair type is "curly hair", the hair color is "red", and the specific face image is attached below."

[0367] The above is a detailed description of the embodiment of the present invention. This system allows users to try out multiple hairstyles at home and choose the best one before visiting a beauty salon. As a result, user satisfaction is improved and communication with the hairdresser is smoother.

[0368] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0369] Step 1:

[0370] Users use a smartphone or dedicated device to capture an image of their face and enter personal information.

[0371] Specific operation: The user activates the smartphone camera and takes a picture of their face, and also inputs their hair type (e.g., curly hair), preferred hairstyle, and color (e.g., red).

[0372] Input: User's face image, hair type, preferred hairstyle and color

[0373] Output: Captured face image and input personal information

[0374] Step 2:

[0375] The facial image and personal information acquired by the terminal are transmitted to the server.

[0376] Specific operation: Using the device's built-in transmission function, the encrypted facial image and personal information are uploaded to the server.

[0377] Input: Captured face image and input personal information

[0378] Output: Facial image and personal information sent to the server

[0379] Step 3:

[0380] The server analyzes the received facial image and personal information.

[0381] How it works: Based on the data received by the server, it uses an AI model and emotion engine to extract facial shape, hair type, skin condition, and emotion data.

[0382] Input: Submitted face image and personal information

[0383] Output: Analyzed feature data (face shape, hair type, skin condition, emotion data)

[0384] Step 4:

[0385] The server generates multiple personalized style candidates based on the extracted feature data.

[0386] How it works: Using an AI model, it generates multiple hairstyles that best fit the user's feature data.

[0387] Input: Parsed feature data

[0388] Output: Image data of style candidates

[0389] Step 5:

[0390] The generated style candidates are sent to the device and presented to the user in real time.

[0391] Specific operation: The server sends image data of the generated style candidates to the terminal, which then displays them on the screen.

[0392] Input: Image data of generated style candidates

[0393] Output: Style suggestions presented to the user

[0394] Step 6:

[0395] The user selects their favorite style from the suggested styles.

[0396] What it does: The user selects the best style from the displayed options and confirms the selection.

[0397] Input: suggested style suggestions

[0398] Output: User selected style

[0399] Step 7:

[0400] The terminal notifies the practitioner of the style results selected by the user.

[0401] Specific operation: The selected style results are sent to the practitioner using the device's built-in notification function.

[0402] Input: User selected style

[0403] Output: Style results communicated to the practitioner

[0404] Step 8:

[0405] The therapist checks the selected style results and the user's emotional data, and begins preparations for the treatment.

[0406] Specific actions: The practitioner prepares for the actual treatment based on the notified style results, and also conducts detailed consultations with the user and provides feedback as needed.

[0407] Input: Notified style results and emotion data

[0408] Output: Prepared treatment and provided feedback

[0409] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0410] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0411] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0412] [Second embodiment]

[0413] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0414] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0415] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0416] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0417] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0418] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0419] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0420] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0421] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0422] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0423] In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0424] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0425] The system of the present invention provides personalized hairstyle suggestions using AI technology to improve the user experience at a beauty salon. This system operates in cooperation with multiple means described below.

[0426] 1. Enter and submit user information

[0427] Terminal

[0428] Users use dedicated terminals in beauty salons to input their individual personal information and real-time facial images. The user captures a facial image using the terminal's camera and inputs information about hair type (e.g., straight or curly), bone structure, and preferred hairstyle and color. All this information is encrypted and sent to a server for further processing.

[0429] 2. Analysis and style candidate generation

[0430] server

[0431] The server receives the user's facial image and personal information sent from the device. Based on this data, an AI model is used to extract characteristic data such as facial shape, hair type, and skin condition, and based on that, optimal hairstyle and color candidates are generated. Multiple style candidates are generated and superimposed on the user's facial image.

[0432] 3. Presentation and selection of style candidates

[0433] Terminal

[0434] The generated style candidates are sent from the server to the device, which then presents the received multiple style candidates to the user in real time, allowing the user to check how the style will look on their actual appearance.

[0435] user

[0436] Users are presented with multiple style options and can choose the one they like best. In some cases, they can also refer to detailed information and comments to help them choose the appropriate style.

[0437] 4. Notification of results and feedback

[0438] Terminal

[0439] The selection results are sent to the hairdresser in real time, who then reviews the style selected by the user and begins preparations for the treatment.

[0440] hairdresser

[0441] Based on the user's chosen style, the stylist will hold a detailed consultation with the user before carrying out any necessary treatments. The stylist will provide feedback on the user's chosen style and make any final adjustments.

[0442] Specific examples

[0443] As a specific example, suppose User A visits a hair salon, captures an image of their face on their device, and enters their hair type as "straight" and their favorite color as "brown." The device sends this information to the server, which uses an AI model to generate multiple brown hairstyles. These styles are sent back to the device superimposed on User A's face image, allowing User A to view them in real time. User A then selects their favorite style, and the result is notified to the hairdresser. After reviewing and consulting with User A, the hairdresser begins treatment based on the selected style.

[0444] In this way, the system enhances communication between users and hairdressers, providing optimal styles tailored to individual needs and improving the salon experience.

[0445] The processing flow will be explained below.

[0446] Step 1:

[0447] The user operates the device, captures a facial image using the camera, and inputs personal information such as hair type, preferred color, and hairstyle.

[0448] Step 2:

[0449] The device sends the captured facial image and input personal information to a server, where the data is encrypted to protect privacy.

[0450] Step 3:

[0451] The server analyzes the received facial image data and personal information, and extracts characteristic data such as the user's facial shape and hair type using an image analysis algorithm.

[0452] Step 4:

[0453] The server inputs the extracted feature data into an AI model to generate multiple personalized hairstyle and color suggestions based on the user's specific information.

[0454] Step 5:

[0455] The server then sends the generated style candidate images to the device in real time, which then displays an image of the user's face with the style applied.

[0456] Step 6:

[0457] The device then presents the received style candidate images to the user, who can then view the style applied to their own face on the device screen.

[0458] Step 7:

[0459] Users can select the style they like best from the suggested styles, and can also view detailed information and comments about the selected style.

[0460] Step 8:

[0461] The device sends the user's selected style results to the server in real time.

[0462] Step 9:

[0463] The server processes the selection results and sends a notification to the hairdresser's device, where the hairdresser can see the user's selection results in real time.

[0464] Step 10:

[0465] After reviewing the user's chosen style, the hairstylist will hold a detailed consultation with the user and provide feedback as needed.

[0466] Step 11:

[0467] Based on the final feedback, the hairdresser will begin the actual treatment and create a hairstyle that meets the customer's wishes.

[0468] Example 1

[0469] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0470] Currently, hair salons have difficulty proposing hairstyles that effectively reflect the individual preferences and characteristics of their customers. Furthermore, there is a lack of means for customers to quickly communicate their chosen style to the hairdresser and incorporate their feedback. As a result, achieving the hairstyle the customer desires often requires time and effort.

[0471] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0472] In this invention, the server includes means for inputting a user's facial image and personal information, means for encrypting the facial image and personal information and transmitting them to the server, means for decrypting the encrypted data and extracting feature data in the server, means for generating a plurality of personalized style candidates using a generative AI model based on the extracted feature data, means for presenting the generated style candidates to the user in real time, and means for notifying the provider of the style results selected by the user. This enables the server to quickly propose an optimal hairstyle according to the user's individual preferences and facilitates smooth communication with the hairdresser.

[0473] A "user's facial image" is video data that includes the facial features of an individual user.

[0474] "Personal Information" refers to information about a user, such as their hair type, bone structure, and preferred styles and colors.

[0475] "Encryption" is the process of using a cryptographic standard (e.g., AES-256) to transmit data securely and converting the content into a format that is unreadable to third parties.

[0476] "Feature data" refers to user-specific data points and attributes extracted based on the user's facial image and personal information.

[0477] A "generative AI model" is a model that uses artificial intelligence technology to generate optimal hairstyle candidates from input data.

[0478] "Style suggestions" are multiple hairstyle options generated by the AI ​​model to suggest to the user.

[0479] "Real-time" refers to processing within a timeframe that provides an immediate response to user interaction.

[0480] A "provider" is a professional who provides a service to a user, such as a hairdresser.

[0481] "Feedback" is any evaluation, opinion, or advice from the provider regarding the style selected by the user.

[0482] This invention is a system for providing personalized hairstyle suggestions that reflect the individual preferences and characteristics of users at beauty salons. The system includes a series of techniques for encrypting data entered by users using a dedicated terminal, analyzing it on a server, and generating optimal style suggestions.

[0483] Hardware and software used

[0484] The server uses NVIDIA GPUs (e.g., NVIDIA RTX 3080) and TensorFlow to run AI models, and the OpenSSL library is used for data encryption and decryption. The device is equipped with a camera, which is used to capture the user's facial image.

[0485] System Operation Overview

[0486] Terminal

[0487] Users use a dedicated terminal at the salon to input their facial image and personal information (e.g., hair type, bone structure, preferred hairstyle and color). Specifically, users capture a facial image using the terminal's camera and input various personal information on the screen. All of this information is encrypted using AES-256 to ensure security. The encrypted data is then sent to a server via the Internet.

[0488] server

[0489] The server receives the encrypted data sent from the device and decrypts it using AES-256. The decrypted data includes the user's facial image and personal information. The server then runs an AI model using TensorFlow to extract the user's feature data from the facial image and personal information. Based on this feature data, the server generates multiple style candidates, including optimal hairstyles and colors. The generated style candidates are then overlaid on the user's facial image.

[0490] Terminal

[0491] The generated style candidates are sent from the server to the device. The device then presents these style candidates to the user in real time, allowing the user to see how they will look when applied to their own face. The user can then select the style candidate they like best from the multiple style candidates presented. If necessary, they can refer to detailed information and comments.

[0492] user

[0493] Users can compare multiple style options displayed in real time and select the hairstyle that best suits them. The results of this selection are then sent to the provider (e.g., hairdresser) in real time.

[0494] provider

[0495] The provider will review the user's selections on the device and begin preparing the necessary treatment. During a detailed consultation with the user, the provider will provide feedback based on the style selected and make any final adjustments.

[0496] Specific examples

[0497] For example, let's say User A visits a hair salon. First, User A uses a device to capture an image of their face, and enters their hair type as "straight" and their preferred color as "brown." This information is encrypted on the device and sent to the server. The server decrypts the encrypted data and uses an AI model to generate multiple brown hairstyle candidates. These generated style candidates are superimposed on User A's face image and sent back to the device. User A compares the displayed multiple style candidates and selects the one they like best. The selection result is notified to the salon provider, who then begins preparations for the treatment and, after a detailed consultation with User A, adjusts the style to the final version.

[0498] Prompt Sentence Examples

[0499] "Run an AI model to suggest the best hairstyle options based on the user's facial image and personal information."

[0500] This system quickly suggests the best hairstyle based on the user's individual preferences and enables smooth communication with the hairdresser, which is expected to improve the user experience at the salon and increase satisfaction.

[0501] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0502] Step 1:

[0503] The user inputs his / her facial image and personal information using a dedicated terminal at the beauty salon.

[0504] Specifically, users use the device's camera to capture a facial image and enter personal information such as hair type, bone structure, and preferred hairstyle and color.

[0505] Input: User's face image and personal information.

[0506] Output: The encrypted data.

[0507] Step 2:

[0508] The device encrypts the entered facial image and personal information using AES-256.

[0509] Specifically, the terminal encrypts input data and executes processes to maintain security.

[0510] Input: User's face image and personal information.

[0511] Output: The encrypted data.

[0512] Step 3:

[0513] The encrypted data is sent to a server over the Internet.

[0514] Specifically, the terminal executes a process of transmitting encrypted data to a server via a network.

[0515] Input: Encrypted data.

[0516] Output: The data sent to the server.

[0517] Step 4:

[0518] The server decrypts the received encrypted data using AES-256.

[0519] Specifically, the server runs a process to decrypt the encrypted data and retrieve the original facial image and personal information.

[0520] Input: Encrypted data.

[0521] Output: Decoded face image and personal information.

[0522] Step 5:

[0523] The server extracts feature data based on the decoded face image and personal information.

[0524] Specifically, the server runs an AI model to extract feature data from the user's facial image and personal information.

[0525] Input: Decoded face image and personal information.

[0526] Output: Extracted feature data.

[0527] Step 6:

[0528] The server generates multiple personalized style candidates using a generative AI model based on the extracted feature data.

[0529] Specifically, the server uses an AI model to generate multiple hairstyle options that are overlaid on the user's face image.

[0530] Input: Extracted feature data.

[0531] Output: Multiple style candidates.

[0532] Step 7:

[0533] The server sends the generated style candidates to the terminal in real time.

[0534] Specifically, a process is executed to transmit the generated style candidate data to the terminal via the Internet.

[0535] Input: Multiple style candidates.

[0536] Output: Data sent to the terminal.

[0537] Step 8:

[0538] The device presents the received style suggestions to the user in real time.

[0539] Specifically, the device executes a process to display style candidates superimposed on the user's facial image.

[0540] Input: Style suggestions received from the server.

[0541] Output: Multiple style suggestions displayed to the user.

[0542] Step 9:

[0543] The user selects the style they like best from the multiple style options presented.

[0544] Specifically, the user compares style candidates on the device screen and makes a selection.

[0545] Input: Several style suggestions displayed on the device.

[0546] Output: User selected style.

[0547] Step 10:

[0548] The device notifies the provider of the user's selected style results in real time.

[0549] Specifically, the terminal executes a process of transmitting the selection results to the provider in real time via the network.

[0550] Input: The style selected by the user.

[0551] Output: The selection results communicated to the provider.

[0552] Step 11:

[0553] The provider will review the user's selections and begin preparing the necessary treatment.

[0554] Specifically, the provider checks the selection results on the terminal and plans the treatment in consultation with the user.

[0555] Input: The selection results notified to the provider.

[0556] Output: Preparing and planning the procedure.

[0557] Step 12:

[0558] The provider will hold a detailed consultation with the user and provide feedback based on the style selected.

[0559] Specifically, the provider will actually have a conversation with the user and make final style adjustments.

[0560] Input: The selection results notified to the provider.

[0561] Output: Feedback and final adjustments provided to the user.

[0562] (Application example 1)

[0563] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0564] In modern society, the expectations and needs of users of beauty services are becoming more diverse. While remote home beauty treatments and virtual salon experiences are on the rise, choosing the hairstyle and color that best suits them can be a challenging task. Furthermore, the time and expense required to actually try on a selected style can reduce user satisfaction. Given this background, there is a demand for a system that allows users to try on various hairstyles in real time from the comfort of their own home and select the style that best suits them.

[0565] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0566] In this invention, the server includes a device for inputting a user's facial image and personal information, a device for analyzing the facial image and personal information and extracting feature data, a device for generating a plurality of personalized style candidates based on the extracted feature data, a device for presenting the generated style candidates to the user in real time, a device for notifying the technician of the style selected by the user, and a device for transmitting the selected style, presenting the style in real time, and generating a prompt message for accepting the selection. This allows the user to try out various hairstyles in real time from the comfort of their own home, select the best style, and immediately notify the technician.

[0567] "Device for inputting a user's facial image and personal information" refers to a terminal or application that allows a user to input their own facial photo and personal information (e.g., hair type, preferred color, etc.).

[0568] "Device that analyzes facial images and personal information and extracts feature data" refers to a server or system that processes input facial images and personal information and uses AI technology to analyze feature data such as facial shape and hair type.

[0569] A "device for generating multiple personalized style candidates" is a system that includes an algorithm or AI model that automatically generates optimal style and hair color candidates for a user based on extracted feature data.

[0570] The "device that presents generated style candidates to the user in real time" refers to a terminal or application that displays a plurality of generated style candidates to the user in real time for confirmation.

[0571] The "device for notifying a technician of the style results selected by a user" refers to a system for notifying a technician (e.g., a hairdresser or hair stylist) of the style information selected by a user in real time.

[0572] "A device that transmits selected styles, presents styles in real time, and generates prompts to accept selections" refers to a device that transmits style information and provides an interface for selection using prompts that are generated based on user selections.

[0573] The system that realizes this application example is composed of a device that inputs a user's facial image and personal information, a server that analyzes the facial image and personal information and extracts feature data, a server that generates personalized style candidates based on the extracted feature data, a terminal that presents the generated style candidates to the user in real time, a device that notifies the engineer of the selected style result, and a device that transmits the selected style and generates a prompt sentence.

[0574] Specific operation of the system

[0575] The operation of this system is as follows.

[0576] 1. Enter and submit user information

[0577] Users use a dedicated device or smartphone app to capture an image of their face and enter personal information such as hair type and preferred color. The data is encrypted and then sent to a server. Logitec webcams and other cameras are used as camera devices, and the Python OpenCV library is used for image processing.

[0578] 2. Analysis and style candidate generation

[0579] The submitted facial image and personal information are received by a server and analyzed using an AI model. The server extracts feature data such as facial shape and hair type, and generates multiple hairstyle candidates based on that data. This AI model is built using deep learning frameworks such as TensorFlow and Keras.

[0580] 3. Presentation and selection of style candidates

[0581] The generated hairstyle candidates are sent from the server to the user's device. The user can view the hairstyles in real time and select the style they like best. This process uses OpenCV image display technology. The selected style is then sent back to the server.

[0582] 4. Notification of results and feedback

[0583] The selection results are notified to the technician in real time, and the technician can use that information to prepare for the procedure. The technician can provide feedback to the user and make final adjustments. A prompt generation device is then used to generate a prompt based on the user's selection and send it to the technician.

[0584] Examples of prompt statements

[0585] A specific example of a prompt is as follows:

[0586] Based on the user's face image and the following information, please provide five suitable hairstyle options.

[0587] Hair Type: Straight

[0588] Favorite color: Brown

[0589] Facial images are provided in the following formats:

[0590] <Image data>

[0591] In this way, users can virtually try out different styles from the comfort of their own home, with the results being communicated to the technician in real time, which is expected to provide a smooth and efficient beauty experience.

[0592] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0593] Step 1:

[0594] Users use a dedicated device or smartphone app to capture an image of their face and enter personal information such as hair type and preferred color. The facial image is captured using a camera device (e.g., a Logitec webcam). The personal information is collected in text format based on the user's input. This data is encrypted and then sent to the server.

[0595] Input: Personal information such as the user's face image, hair type, favorite color, etc.

[0596] Output: Encrypted face image and personal information are sent to the server

[0597] Step 2:

[0598] The server analyzes the received facial image and personal information and extracts feature data from them. Specifically, it uses image analysis technology (e.g., OpenCV) to detect facial shape, hair type, skin condition, etc. It also analyzes the input personal information to extract information about preferred styles.

[0599] Input: Encrypted face image and personal information

[0600] Output: Feature data such as face shape, hair texture, skin condition, etc.

[0601] Step 3:

[0602] The server generates personalized hairstyle candidates based on the extracted feature data. To do this, it uses a generative AI model (e.g., a deep learning model using TensorFlow or Keras). This model generates multiple hairstyles that match the user's features.

[0603] Input: feature data

[0604] Output: Multiple personalized hairstyle suggestions

[0605] Step 4:

[0606] The server sends the generated hairstyle candidates to the user's device, which then presents them to the user in real time. Each proposed style is combined with the facial image, and the user can view the style displayed through the interface. This process uses image display technology (e.g., OpenCV).

[0607] Input: Hairstyle candidate

[0608] Output: Hairstyle suggestions presented in real time

[0609] Step 5:

[0610] The user selects the hairstyle they like best from the multiple hairstyle options presented, and the selection is sent back to the server from the device.

[0611] Input: User selection

[0612] Output: The selection results are sent to the server

[0613] Step 6:

[0614] The server notifies the technician of the user's selection. The technician uses this information to prepare for the treatment. Furthermore, a prompt message based on the user's selection is sent to the technician. This allows the technician to respond quickly to the user's request.

[0615] Input: User selection

[0616] Output: Generates and sends notifications and prompts to technicians

[0617] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0618] The system of the present invention provides personalized hairstyle suggestions by combining AI technology and an emotion engine to improve the user experience in a beauty salon. This system operates in cooperation with multiple means described below.

[0619] 1. Enter and submit user information

[0620] Terminal

[0621] The user uses a dedicated terminal at the salon to input their personal information and real-time facial image. The user captures the facial image using the terminal's camera, and the user's hair type (e.g., straight or curly), bone structure, preferred hairstyle and color, and the emotion engine reads the user's emotions from their facial expressions. All of this information is encrypted and sent to the server.

[0622] 2. Analysis and style candidate generation

[0623] server

[0624] The server receives the user's facial image, personal information, and emotional data sent from the device. Based on this data, an AI model is used to extract feature data such as facial shape, hair type, skin condition, and emotional data obtained by an emotion engine, and based on this, optimal hairstyle and color candidates are generated. Multiple style candidates are generated and superimposed on the user's facial image.

[0625] 3. Presentation and selection of style candidates

[0626] Terminal

[0627] The generated style candidates are sent from the server to the device, which then presents the received multiple style candidates to the user in real time, allowing the user to check the style applied to their own face.

[0628] user

[0629] Users can choose the style they like best from a selection of suggested styles. In some cases, they can refer to detailed information and comments to select the appropriate style. Additionally, the system analyzes users' emotions in real time and reflects this information in the style selection.

[0630] 4. Notification of results and feedback

[0631] Terminal

[0632] The selection results are sent to the hairdresser in real time, who then reviews the style selected by the user and begins preparations for the treatment.

[0633] hairdresser

[0634] Based on the user's selections and emotional data, the hairdresser will hold a detailed consultation with the user and provide feedback as needed.

[0635] Specific examples

[0636] As a specific example, suppose User B visits a hair salon, captures an image of their face on their device, and enters their hair type as "curly" and their favorite color as "blonde." The device sends this information, along with the user's emotional data, to the server. The server uses an AI model and an emotion engine to generate multiple blonde hairstyles, which are then superimposed on User B's facial image and sent back to the device. User B reviews the style options presented in real time and selects the most suitable one. The emotional data is also reflected, and the user's selected style is notified to the hairdresser. The hairdresser reviews the selection and emotional data, prepares for the treatment, consults with User B, and makes any final adjustments before beginning the treatment.

[0637] In this way, by incorporating the user's emotional data, this system can provide more personalized style suggestions, improve user satisfaction, and enhance communication with the hairdresser.

[0638] The processing flow will be explained below.

[0639] Step 1:

[0640] The user operates the device, captures a facial image using the camera, and inputs personal information such as hair type, preferred color, and hairstyle. The emotion engine then analyzes the user's facial expressions and collects emotional data.

[0641] Step 2:

[0642] The device transmits the captured facial image, input personal information, and acquired emotion data to a server, where the data is encrypted to protect privacy.

[0643] Step 3:

[0644] The server analyzes the received facial image data, personal information, and emotional data, and extracts feature data based on the user's facial shape, hair type, skin condition, and emotional data using image analysis and emotional analysis algorithms.

[0645] Step 4:

[0646] The server inputs the extracted feature data and emotion data into an AI model to generate multiple personalized hairstyle and color suggestions based on the user's unique information and current emotion. Emotion data is given special weight when generating style suggestions.

[0647] Step 5:

[0648] The server sends the generated style candidate images to the device in real time, and the device displays the received style candidates superimposed on the user's facial image, allowing the user to check them in real time.

[0649] Step 6:

[0650] The device presents style candidates to the user, who then sees the style applied to their face. The proposed style takes into account the user's current emotional state, making it highly appropriate.

[0651] Step 7:

[0652] The user selects the style they like best from the multiple style options presented, and the selection reflects the user's emotional state, increasing satisfaction.

[0653] Step 8:

[0654] The device transmits the user's selected style results, including emotional data, to the server in real time.

[0655] Step 9:

[0656] The server processes the selection results and emotion data and sends a notification to the hairdresser's device, where the hairdresser can check the selection results and emotion data in real time.

[0657] Step 10:

[0658] The hairstylist will then hold an in-depth consultation with the user based on their selection and emotional data, providing feedback that takes their emotions into account and suggesting tweaks to the style if necessary.

[0659] Step 11:

[0660] Based on the final feedback, the hairdresser will begin the actual treatment and achieve the hairstyle that matches the user's wishes. Emotional data is taken into consideration throughout the treatment, improving the user experience.

[0661] In this way, by adding emotional data, this system can provide more personalized style suggestions, improve user satisfaction, and enhance communication with hairdressers.

[0662] Example 2

[0663] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0664] To improve customer satisfaction at hair salons, personalized style suggestions that take into account the individual characteristics and emotions of each user are required. However, conventional systems only consider basic information such as the user's facial image, hair type, and preferred style, and are unable to suggest styles that reflect detailed personal data such as the user's emotions and skin condition. This results in issues such as not being able to meet user expectations and not being able to communicate effectively with the hairdresser.

[0665] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0666] In this invention, the server includes a means for acquiring the user's emotional data and reflecting it in the creation of a style, a means for proposing a style and color based on the user's skin condition, and a means for the hairdresser to provide feedback on the result of the user's selection from the presented style candidates. This makes it possible to propose a style based on detailed personal data such as the user's facial image, hair type, and preferred style, as well as their emotions and skin condition.

[0667] A "facial image" is a digital image that captures the visual features of a user's face.

[0668] "Personal information" refers to information about the user, such as hair type, preferred hairstyle and color.

[0669] "Feature data" refers to characteristic data such as facial shape, hair type, and skin condition extracted from the user's facial image and personal information.

[0670] "Style candidates" are multiple hairstyle and color suggestions generated based on feature data, and are presented as a composite image superimposed on the user's facial image.

[0671] An "emotion engine" is software or hardware that reads emotions from a user's facial expressions and analyzes them as data.

[0672] "Real-time" refers to a state in which data and processing occur with almost no delay, allowing users and systems to respond immediately.

[0673] "Feedback" refers to any comments or information provided by a Hairdresser regarding improvements or evaluation of a User's selection.

[0674] "Suggestions" are specific style and color options and ideas to present to users.

[0675] The system of the present invention provides personalized hairstyle suggestions by combining AI technology and an emotion engine to improve the user experience in a beauty salon. This system utilizes the user's facial image, personal information, and emotion data, and includes a series of processes to generate and present optimal style suggestions in real time.

[0676] First, the user uses a dedicated terminal installed in the salon to input their individual personal information and a real-time facial image. The terminal is equipped with a camera, and the user presses the "take a photo" button to capture a facial image. Along with this facial image, the user also inputs their hair type, preferred hairstyle and color. In addition, the emotion engine installed in the terminal reads emotions from the user's facial expressions, and this information is also collected. This data is encrypted and sent to the server.

[0677] The server analyzes the received facial image, personal information, and emotional data. First, it uses a facial recognition algorithm to extract the shape and features of the face. It then analyzes the acquired emotional data using an emotion engine to recognize specific emotions. Based on this analyzed data, the server uses an AI model to generate optimal hairstyle and color candidates. Multiple style candidates are generated and superimposed on the user's facial image.

[0678] The generated style candidates are then sent from the server to the device. The device then presents the received style candidates to the user in real time. The user can then select the style they like best from the multiple style candidates presented. When making a selection, the user can refer to detailed information and comments for each style. In addition, the user's emotions are analyzed in real time, and this information is reflected in the style selection.

[0679] Once the user selects the best style, the results are communicated to the hairdresser in real time. The hairdresser then conducts a detailed consultation with the user based on the selected style and emotional data, providing feedback as needed. The hairdresser then begins preparations for the treatment, making any final adjustments before styling the user's hair.

[0680] As a concrete example, suppose User B visits a hair salon, captures a facial image on a device, and inputs "hair type: curly" and "preferred color: blonde." This data and emotional data are sent to the server. The server uses an AI model and an emotion engine to generate a blonde style for User B and overlays it on the user's facial image. The generated style candidates are displayed on the device, and User B selects the most suitable style. The selection result, reflecting User B's emotions, is notified to the hairdresser, who makes preparations based on that information and begins the treatment after a final consultation.

[0681] An example prompt is:

[0682] "I'm a woman in her 20s with a cheerful face and curly hair. My favorite color is blonde. Please generate the best hairstyle for her."

[0683] By incorporating user emotional data, the system can provide personalized style suggestions, improve user satisfaction, and enhance communication with hairdressers.

[0684] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0685] Step 1: Enter your user information

[0686] Terminal

[0687] The user uses a dedicated terminal at the salon and first enters their personal information. The terminal is equipped with a camera, and the user captures a facial image by facing the camera and pressing the "shoot" button. Specifically, the facial image (input) is obtained and the data is temporarily saved (output). The user also inputs their hair type (e.g. straight hair, curly hair), preferred hairstyle and color (input) through the UI. Along with this information, the emotion engine analyzes facial expressions and obtains emotional data (output).

[0688] Step 2: Submit user information

[0689] Terminal

[0690] The captured facial image, input hair type, preferred hairstyle and color, and acquired emotion data are all encrypted. The encrypted data packet is sent to the server. The input is the entire user information, and the output is the encrypted data packet.

[0691] Step 3: Receiving and analyzing user information

[0692] server

[0693] The server receives encrypted data packets from the device (input). The received data is decrypted and separated into facial images, personal information, and emotional data (output). The facial images are then analyzed using a facial recognition algorithm to extract facial shapes and features, generating feature data (output). The emotion engine then analyzes the acquired emotional data to identify specific emotions (e.g., joy, surprise, sadness).

[0694] Step 4: Generate style candidates

[0695] server

[0696] The server operates a generative AI model based on the analyzed feature data and emotional data (input). This AI model combines the user's face shape, hair type, personal information, and emotional data to generate multiple candidates for optimal hairstyles and colors (output). It also creates a composite image in which these styles are superimposed on the user's face image. Style candidate data including this composite image is generated.

[0697] Step 5: Submit and present your style suggestions

[0698] server

[0699] The generated style candidates and the composite image are sent to the terminal (the input is the style candidate data, and the output is the sent data). This is also encrypted and transmitted securely.

[0700] Terminal

[0701] The device decodes the received style candidate data (input) and presents multiple style candidates and their detailed information to the user in real time (output). The user can check each style through the UI. For example, pressing the "Next" button displays the next style candidate.

[0702] Step 6: User Style Selection

[0703] user

[0704] The user selects the style they like best from the multiple style candidates presented (input: user selection). By pressing the "Select" button on the UI, that style is selected (output). If necessary, detailed information and comments for each style can be viewed to help with the final decision. Additionally, the emotion engine analyzes the user's facial expressions while selecting and reflects them in the selection results.

[0705] Step 7: Notification of results and feedback

[0706] Terminal

[0707] The server notifies the user of the style they have selected in real time (the input is the selection result, and the output is the notification data). This information is then forwarded to the hairdresser's device.

[0708] hairdresser

[0709] The hairdresser confirms the user's selection and emotional data (input) and begins preparations for the treatment. He / she then consults with the user (output) and provides necessary feedback, such as explaining the feasibility of the selected style and the necessary treatments. After making any final adjustments, the treatment begins.

[0710] Through this series of processes, personalized style suggestions and high-quality treatments are realized based on the user's emotions and personal information.

[0711] (Application example 2)

[0712] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0713] Hair salons need to propose optimal hairstyles based on the individual feelings and preferences of their customers, thereby increasing customer satisfaction. However, conventional systems have the problem that customers cannot confirm which style suits them until they actually visit the salon. Furthermore, there is no way for users to try the style at home before the treatment, and due to a lack of communication with the hairdresser, the desired style may not be fully reflected.

[0714] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0715] In this invention, the server includes: means for inputting a user's facial image and personal information; means for analyzing the facial image and personal information and extracting feature data; means for generating multiple personalized style candidates based on the extracted feature data; means for presenting the generated style candidates to the user in real time; means for notifying the hairdresser of the user's selected style; means for the user to use the virtual beauty salon app at home; means for capturing a facial image and inputting a desired hairstyle using a smartphone; and means for generating a hairstyle in real time using an AI model and an emotion engine through the virtual beauty salon app. This allows the user to try out multiple hairstyles at home and select the best style before visiting the salon. As a result, user satisfaction is improved and communication with the hairdresser is smoother.

[0716] "User" refers to an individual who uses the services of a beauty salon.

[0717] "Facial image" means a photograph or image data of a user's face.

[0718] "Personal Information" refers to your hair type, preferred hairstyles and colors, and other personal information.

[0719] "Feature data" refers to data such as facial shape, hair type, and skin condition analyzed from the input facial image and personal information.

[0720] "Style suggestions" refers to multiple hairstyles suggested to the user.

[0721] "Real-time" means immediate response or processing without delay.

[0722] "Practitioner" refers to a professional who performs hairstyles in a beauty salon, i.e., a hairdresser.

[0723] A "virtual beauty salon app" refers to application software that allows users to try out hairstyles at home using their smartphones.

[0724] "Smartphone" means a portable communication device that has Internet connectivity and can be used by installing applications.

[0725] An "AI model" refers to artificial intelligence that uses technologies such as machine learning and deep learning to analyze a user's characteristic data and generate optimal style candidates.

[0726] "Emotion engine" refers to technology that reads emotions from a user's facial expressions and uses those emotions as data for analysis.

[0727] The system of the present invention provides personalized hairstyle suggestions that combine AI technology and an emotion engine to improve the user experience in beauty salons. This system includes a means for inputting a user's facial image and personal information, a means for analyzing the facial image and personal information and extracting feature data, a means for generating multiple personalized style candidates based on the extracted feature data, a means for presenting the generated style candidates to the user in real time, and a means for notifying the hairdresser of the style results selected by the user.

[0728] 1. Enter and submit user information

[0729] Terminal

[0730] Users use their smartphone to capture a photo of their face at home or at a dedicated terminal in a beauty salon, and input their hair type, preferred hairstyle, and color. The facial image is then captured using the smartphone's camera or the dedicated terminal's camera. This data is encrypted as needed and sent to a server.

[0731] 2. Analysis and style candidate generation

[0732] server

[0733] After receiving the user's personal information and facial image, the server uses an AI model to analyze facial shape, hair type, skin condition, and even emotional data obtained from an emotion engine to extract feature data. Multiple style candidates are then generated based on the extracted feature data. The generated style candidates are then overlaid on the user's facial image to present a realistic image.

[0734] 3. Presentation and selection of style candidates

[0735] Terminal

[0736] The generated style candidates are sent from the server to the device and are presented to the user in real time on a smartphone or dedicated device. The user can then review multiple style candidates applied to their face and select the one that best suits them.

[0737] user

[0738] Users can select their favorite style from the suggested styles, and the system will also incorporate suggestions based on emotional data to optimize the selection.

[0739] 4. Notification of results and feedback

[0740] Terminal

[0741] The results of the selected style are notified to the practitioner in real time, who then confirms the style selected by the user and begins preparations for the treatment.

[0742] Practitioner

[0743] The practitioner will conduct a detailed consultation based on the user's selections, provide feedback as needed, and make any final adjustments before beginning the treatment.

[0744] Hardware and software used

[0745] In this invention, in addition to the camera of a smartphone or dedicated device, a server is used to run the AI ​​model, and existing AI services such as Azure Face API, Google Cloud Vision, and Amazon Rekognition are used for the analysis.

[0746] Specific examples

[0747] For example, when User A uses this system at home, they capture a facial image using their smartphone and enter their hair type as "curly" and color as "red." The smartphone encrypts this information and sends it to the server. The server-side AI model analyzes this data, generates multiple personalized style options for User A, and sends the results to the smartphone. Once User A selects the optimal style, the results are notified to the hairdresser in real time.

[0748] Prompt Sentence Examples

[0749] "Generate the best hairstyle candidates for the user based on their face image and preferred hairstyle information. The hair type is "curly hair", the hair color is "red", and the specific face image is attached below."

[0750] The above is a detailed description of the embodiment of the present invention. This system allows users to try out multiple hairstyles at home and choose the best one before visiting a beauty salon. As a result, user satisfaction is improved and communication with the hairdresser is smoother.

[0751] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0752] Step 1:

[0753] Users use a smartphone or dedicated device to capture an image of their face and enter personal information.

[0754] Specific operation: The user activates the smartphone camera and takes a picture of their face, and also inputs their hair type (e.g., curly hair), preferred hairstyle, and color (e.g., red).

[0755] Input: User's face image, hair type, preferred hairstyle and color

[0756] Output: Captured face image and input personal information

[0757] Step 2:

[0758] The facial image and personal information acquired by the terminal are transmitted to the server.

[0759] Specific operation: Using the device's built-in transmission function, the encrypted facial image and personal information are uploaded to the server.

[0760] Input: Captured face image and input personal information

[0761] Output: Facial image and personal information sent to the server

[0762] Step 3:

[0763] The server analyzes the received facial image and personal information.

[0764] How it works: Based on the data received by the server, it uses an AI model and emotion engine to extract facial shape, hair type, skin condition, and emotion data.

[0765] Input: Submitted face image and personal information

[0766] Output: Analyzed feature data (face shape, hair type, skin condition, emotion data)

[0767] Step 4:

[0768] The server generates multiple personalized style candidates based on the extracted feature data.

[0769] How it works: Using an AI model, it generates multiple hairstyles that best fit the user's feature data.

[0770] Input: Parsed feature data

[0771] Output: Image data of style candidates

[0772] Step 5:

[0773] The generated style candidates are sent to the device and presented to the user in real time.

[0774] Specific operation: The server sends image data of the generated style candidates to the terminal, which then displays them on the screen.

[0775] Input: Image data of generated style candidates

[0776] Output: Style suggestions presented to the user

[0777] Step 6:

[0778] The user selects their favorite style from the suggested styles.

[0779] What it does: The user selects the best style from the displayed options and confirms the selection.

[0780] Input: suggested style suggestions

[0781] Output: User selected style

[0782] Step 7:

[0783] The terminal notifies the practitioner of the style results selected by the user.

[0784] Specific operation: The selected style results are sent to the practitioner using the device's built-in notification function.

[0785] Input: User selected style

[0786] Output: Style results communicated to the practitioner

[0787] Step 8:

[0788] The therapist checks the selected style results and the user's emotional data, and begins preparations for the treatment.

[0789] Specific actions: The practitioner prepares for the actual treatment based on the notified style results, and also conducts detailed consultations with the user and provides feedback as needed.

[0790] Input: Notified style results and emotion data

[0791] Output: Prepared treatment and provided feedback

[0792] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0793] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0794] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[0795] [Third embodiment]

[0796] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[0797] 5, the data processing system 310 includes the data processing device 12 and a headset type terminal 314. An example of the data processing device 12 is a server.

[0798] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0799] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[0800] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0801] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0802] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0803] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0804] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0805] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0806] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0807] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[0808] The system of the present invention provides personalized hairstyle suggestions using AI technology to improve the user experience at a beauty salon. This system operates in cooperation with multiple means described below.

[0809] 1. Enter and submit user information

[0810] Terminal

[0811] Users use dedicated terminals in beauty salons to input their individual personal information and real-time facial images. The user captures a facial image using the terminal's camera and inputs information about hair type (e.g., straight or curly), bone structure, and preferred hairstyle and color. All this information is encrypted and sent to a server for further processing.

[0812] 2. Analysis and style candidate generation

[0813] server

[0814] The server receives the user's facial image and personal information sent from the device. Based on this data, an AI model is used to extract characteristic data such as facial shape, hair type, and skin condition, and based on that, optimal hairstyle and color candidates are generated. Multiple style candidates are generated and superimposed on the user's facial image.

[0815] 3. Presentation and selection of style candidates

[0816] Terminal

[0817] The generated style candidates are sent from the server to the device, which then presents the received multiple style candidates to the user in real time, allowing the user to check how the style will look on their actual appearance.

[0818] user

[0819] Users are presented with multiple style options and can choose the one they like best. In some cases, they can also refer to detailed information and comments to help them choose the appropriate style.

[0820] 4. Notification of results and feedback

[0821] Terminal

[0822] The selection results are sent to the hairdresser in real time, who then reviews the style selected by the user and begins preparations for the treatment.

[0823] hairdresser

[0824] Based on the user's chosen style, the stylist will hold a detailed consultation with the user before carrying out any necessary treatments. The stylist will provide feedback on the user's chosen style and make any final adjustments.

[0825] Specific examples

[0826] As a specific example, suppose User A visits a hair salon, captures an image of their face on their device, and enters their hair type as "straight" and their favorite color as "brown." The device sends this information to the server, which uses an AI model to generate multiple brown hairstyles. These styles are sent back to the device superimposed on User A's face image, allowing User A to view them in real time. User A then selects their favorite style, and the result is notified to the hairdresser. After reviewing and consulting with User A, the hairdresser begins treatment based on the selected style.

[0827] In this way, the system enhances communication between users and hairdressers, providing optimal styles tailored to individual needs and improving the salon experience.

[0828] The processing flow will be explained below.

[0829] Step 1:

[0830] The user operates the device, captures a facial image using the camera, and inputs personal information such as hair type, preferred color, and hairstyle.

[0831] Step 2:

[0832] The device sends the captured facial image and input personal information to a server, where the data is encrypted to protect privacy.

[0833] Step 3:

[0834] The server analyzes the received facial image data and personal information, and extracts characteristic data such as the user's facial shape and hair type using an image analysis algorithm.

[0835] Step 4:

[0836] The server inputs the extracted feature data into an AI model to generate multiple personalized hairstyle and color suggestions based on the user's specific information.

[0837] Step 5:

[0838] The server then sends the generated style candidate images to the device in real time, which then displays an image of the user's face with the style applied.

[0839] Step 6:

[0840] The device then presents the received style candidate images to the user, who can then view the style applied to their own face on the device screen.

[0841] Step 7:

[0842] Users can select the style they like best from the suggested styles, and can also view detailed information and comments about the selected style.

[0843] Step 8:

[0844] The device sends the user's selected style results to the server in real time.

[0845] Step 9:

[0846] The server processes the selection results and sends a notification to the hairdresser's device, where the hairdresser can see the user's selection results in real time.

[0847] Step 10:

[0848] After reviewing the user's chosen style, the hairstylist will hold a detailed consultation with the user and provide feedback as needed.

[0849] Step 11:

[0850] Based on the final feedback, the hairdresser will begin the actual treatment and create a hairstyle that meets the customer's wishes.

[0851] Example 1

[0852] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[0853] Currently, hair salons have difficulty proposing hairstyles that effectively reflect the individual preferences and characteristics of their customers. Furthermore, there is a lack of means for customers to quickly communicate their chosen style to the hairdresser and incorporate their feedback. As a result, achieving the hairstyle the customer desires often requires time and effort.

[0854] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0855] In this invention, the server includes means for inputting a user's facial image and personal information, means for encrypting the facial image and personal information and transmitting them to the server, means for decrypting the encrypted data and extracting feature data in the server, means for generating a plurality of personalized style candidates using a generative AI model based on the extracted feature data, means for presenting the generated style candidates to the user in real time, and means for notifying the provider of the style results selected by the user. This enables the server to quickly propose an optimal hairstyle according to the user's individual preferences and facilitates smooth communication with the hairdresser.

[0856] A "user's facial image" is video data that includes the facial features of an individual user.

[0857] "Personal Information" refers to information about a user, such as their hair type, bone structure, and preferred styles and colors.

[0858] "Encryption" is the process of using a cryptographic standard (e.g., AES-256) to transmit data securely and converting the content into a format that is unreadable to third parties.

[0859] "Feature data" refers to user-specific data points and attributes extracted based on the user's facial image and personal information.

[0860] A "generative AI model" is a model that uses artificial intelligence technology to generate optimal hairstyle candidates from input data.

[0861] "Style suggestions" are multiple hairstyle options generated by the AI ​​model to suggest to the user.

[0862] "Real-time" refers to processing within a timeframe that provides an immediate response to user interaction.

[0863] A "provider" is a professional who provides a service to a user, such as a hairdresser.

[0864] "Feedback" is any evaluation, opinion, or advice from the provider regarding the style selected by the user.

[0865] This invention is a system for providing personalized hairstyle suggestions that reflect the individual preferences and characteristics of users at beauty salons. The system includes a series of techniques for encrypting data entered by users using a dedicated terminal, analyzing it on a server, and generating optimal style suggestions.

[0866] Hardware and software used

[0867] The server uses NVIDIA GPUs (e.g., NVIDIA RTX 3080) and TensorFlow to run AI models, and the OpenSSL library is used for data encryption and decryption. The device is equipped with a camera, which is used to capture the user's facial image.

[0868] System Operation Overview

[0869] Terminal

[0870] Users use a dedicated terminal at the salon to input their facial image and personal information (e.g., hair type, bone structure, preferred hairstyle and color). Specifically, users capture a facial image using the terminal's camera and input various personal information on the screen. All of this information is encrypted using AES-256 to ensure security. The encrypted data is then sent to a server via the Internet.

[0871] server

[0872] The server receives the encrypted data sent from the device and decrypts it using AES-256. The decrypted data includes the user's facial image and personal information. The server then runs an AI model using TensorFlow to extract the user's feature data from the facial image and personal information. Based on this feature data, the server generates multiple style candidates, including optimal hairstyles and colors. The generated style candidates are then overlaid on the user's facial image.

[0873] Terminal

[0874] The generated style candidates are sent from the server to the device. The device then presents these style candidates to the user in real time, allowing the user to see how they will look when applied to their own face. The user can then select the style candidate they like best from the multiple style candidates presented. If necessary, they can refer to detailed information and comments.

[0875] user

[0876] Users can compare multiple style options displayed in real time and select the hairstyle that best suits them. The results of this selection are then sent to the provider (e.g., hairdresser) in real time.

[0877] provider

[0878] The provider will review the user's selections on the device and begin preparing the necessary treatment. During a detailed consultation with the user, the provider will provide feedback based on the style selected and make any final adjustments.

[0879] Specific examples

[0880] For example, let's say User A visits a hair salon. First, User A uses a device to capture an image of their face, and enters their hair type as "straight" and their preferred color as "brown." This information is encrypted on the device and sent to the server. The server decrypts the encrypted data and uses an AI model to generate multiple brown hairstyle candidates. These generated style candidates are superimposed on User A's face image and sent back to the device. User A compares the displayed multiple style candidates and selects the one they like best. The selection result is notified to the salon provider, who then begins preparations for the treatment and, after a detailed consultation with User A, adjusts the style to the final version.

[0881] Prompt Sentence Examples

[0882] "Run an AI model to suggest the best hairstyle options based on the user's facial image and personal information."

[0883] This system quickly suggests the best hairstyle based on the user's individual preferences and enables smooth communication with the hairdresser, which is expected to improve the user experience at the salon and increase satisfaction.

[0884] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0885] Step 1:

[0886] The user inputs his / her facial image and personal information using a dedicated terminal at the beauty salon.

[0887] Specifically, users use the device's camera to capture a facial image and enter personal information such as hair type, bone structure, and preferred hairstyle and color.

[0888] Input: User's face image and personal information.

[0889] Output: The encrypted data.

[0890] Step 2:

[0891] The device encrypts the entered facial image and personal information using AES-256.

[0892] Specifically, the terminal encrypts input data and executes processes to maintain security.

[0893] Input: User's face image and personal information.

[0894] Output: The encrypted data.

[0895] Step 3:

[0896] The encrypted data is sent to a server over the Internet.

[0897] Specifically, the terminal executes a process of transmitting encrypted data to a server via a network.

[0898] Input: Encrypted data.

[0899] Output: The data sent to the server.

[0900] Step 4:

[0901] The server decrypts the received encrypted data using AES-256.

[0902] Specifically, the server runs a process to decrypt the encrypted data and retrieve the original facial image and personal information.

[0903] Input: Encrypted data.

[0904] Output: Decoded face image and personal information.

[0905] Step 5:

[0906] The server extracts feature data based on the decoded face image and personal information.

[0907] Specifically, the server runs an AI model to extract feature data from the user's facial image and personal information.

[0908] Input: Decoded face image and personal information.

[0909] Output: Extracted feature data.

[0910] Step 6:

[0911] The server generates multiple personalized style candidates using a generative AI model based on the extracted feature data.

[0912] Specifically, the server uses an AI model to generate multiple hairstyle options that are overlaid on the user's face image.

[0913] Input: Extracted feature data.

[0914] Output: Multiple style candidates.

[0915] Step 7:

[0916] The server sends the generated style candidates to the terminal in real time.

[0917] Specifically, a process is executed to transmit the generated style candidate data to the terminal via the Internet.

[0918] Input: Multiple style candidates.

[0919] Output: Data sent to the terminal.

[0920] Step 8:

[0921] The device presents the received style suggestions to the user in real time.

[0922] Specifically, the device executes a process to display style candidates superimposed on the user's facial image.

[0923] Input: Style suggestions received from the server.

[0924] Output: Multiple style suggestions displayed to the user.

[0925] Step 9:

[0926] The user selects the style they like best from the multiple style options presented.

[0927] Specifically, the user compares style candidates on the device screen and makes a selection.

[0928] Input: Several style suggestions displayed on the device.

[0929] Output: User selected style.

[0930] Step 10:

[0931] The device notifies the provider of the user's selected style results in real time.

[0932] Specifically, the terminal executes a process of transmitting the selection results to the provider in real time via the network.

[0933] Input: The style selected by the user.

[0934] Output: The selection results communicated to the provider.

[0935] Step 11:

[0936] The provider will review the user's selections and begin preparing the necessary treatment.

[0937] Specifically, the provider checks the selection results on the terminal and plans the treatment in consultation with the user.

[0938] Input: The selection results notified to the provider.

[0939] Output: Preparing and planning the procedure.

[0940] Step 12:

[0941] The provider will hold a detailed consultation with the user and provide feedback based on the style selected.

[0942] Specifically, the provider will actually have a conversation with the user and make final style adjustments.

[0943] Input: The selection results notified to the provider.

[0944] Output: Feedback and final adjustments provided to the user.

[0945] (Application example 1)

[0946] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[0947] In modern society, the expectations and needs of users of beauty services are becoming more diverse. While remote home beauty treatments and virtual salon experiences are on the rise, choosing the hairstyle and color that best suits them can be a challenging task. Furthermore, the time and expense required to actually try on a selected style can reduce user satisfaction. Given this background, there is a demand for a system that allows users to try on various hairstyles in real time from the comfort of their own home and select the style that best suits them.

[0948] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0949] In this invention, the server includes a device for inputting a user's facial image and personal information, a device for analyzing the facial image and personal information and extracting feature data, a device for generating a plurality of personalized style candidates based on the extracted feature data, a device for presenting the generated style candidates to the user in real time, a device for notifying the technician of the style selected by the user, and a device for transmitting the selected style, presenting the style in real time, and generating a prompt message for accepting the selection. This allows the user to try out various hairstyles in real time from the comfort of their own home, select the best style, and immediately notify the technician.

[0950] "Device for inputting a user's facial image and personal information" refers to a terminal or application that allows a user to input their own facial photo and personal information (e.g., hair type, preferred color, etc.).

[0951] "Device that analyzes facial images and personal information and extracts feature data" refers to a server or system that processes input facial images and personal information and uses AI technology to analyze feature data such as facial shape and hair type.

[0952] A "device for generating multiple personalized style candidates" is a system that includes an algorithm or AI model that automatically generates optimal style and hair color candidates for a user based on extracted feature data.

[0953] The "device that presents generated style candidates to the user in real time" refers to a terminal or application that displays a plurality of generated style candidates to the user in real time for confirmation.

[0954] The "device for notifying a technician of the style results selected by a user" refers to a system for notifying a technician (e.g., a hairdresser or hair stylist) of the style information selected by a user in real time.

[0955] "A device that transmits selected styles, presents styles in real time, and generates prompts to accept selections" refers to a device that transmits style information and provides an interface for selection using prompts that are generated based on user selections.

[0956] The system that realizes this application example is composed of a device that inputs a user's facial image and personal information, a server that analyzes the facial image and personal information and extracts feature data, a server that generates personalized style candidates based on the extracted feature data, a terminal that presents the generated style candidates to the user in real time, a device that notifies the engineer of the selected style result, and a device that transmits the selected style and generates a prompt sentence.

[0957] Specific operation of the system

[0958] The operation of this system is as follows.

[0959] 1. Enter and submit user information

[0960] Users use a dedicated device or smartphone app to capture an image of their face and enter personal information such as hair type and preferred color. The data is encrypted and then sent to a server. Logitec webcams and other cameras are used as camera devices, and the Python OpenCV library is used for image processing.

[0961] 2. Analysis and style candidate generation

[0962] The submitted facial image and personal information are received by a server and analyzed using an AI model. The server extracts feature data such as facial shape and hair type, and generates multiple hairstyle candidates based on that data. This AI model is built using deep learning frameworks such as TensorFlow and Keras.

[0963] 3. Presentation and selection of style candidates

[0964] The generated hairstyle candidates are sent from the server to the user's device. The user can view the hairstyles in real time and select the style they like best. This process uses OpenCV image display technology. The selected style is then sent back to the server.

[0965] 4. Notification of results and feedback

[0966] The selection results are notified to the technician in real time, and the technician can use that information to prepare for the procedure. The technician can provide feedback to the user and make final adjustments. A prompt generation device is then used to generate a prompt based on the user's selection and send it to the technician.

[0967] Examples of prompt statements

[0968] A specific example of a prompt is as follows:

[0969] Based on the user's face image and the following information, please provide five suitable hairstyle options.

[0970] Hair Type: Straight

[0971] Favorite color: Brown

[0972] Facial images are provided in the following formats:

[0973] <Image data>

[0974] In this way, users can virtually try out different styles from the comfort of their own home, with the results communicated to the technician in real time, which is expected to provide a smooth and efficient beauty experience.

[0975] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0976] Step 1:

[0977] Users use a dedicated device or smartphone app to capture an image of their face and enter personal information such as hair type and preferred color. The facial image is captured using a camera device (e.g., a Logitec webcam). The personal information is collected in text format based on the user's input. This data is encrypted and then sent to the server.

[0978] Input: Personal information such as the user's face image, hair type, favorite color, etc.

[0979] Output: Encrypted face image and personal information are sent to the server

[0980] Step 2:

[0981] The server analyzes the received facial image and personal information and extracts feature data from them. Specifically, it uses image analysis technology (e.g., OpenCV) to detect facial shape, hair type, skin condition, etc. It also analyzes the input personal information to extract information about preferred styles.

[0982] Input: Encrypted face image and personal information

[0983] Output: Feature data such as face shape, hair texture, skin condition, etc.

[0984] Step 3:

[0985] The server generates personalized hairstyle candidates based on the extracted feature data. To do this, it uses a generative AI model (e.g., a deep learning model using TensorFlow or Keras). This model generates multiple hairstyles that match the user's features.

[0986] Input: feature data

[0987] Output: Multiple personalized hairstyle suggestions

[0988] Step 4:

[0989] The server sends the generated hairstyle candidates to the user's device, which then presents them to the user in real time. Each proposed style is combined with the facial image, and the user can view the style displayed through the interface. This process uses image display technology (e.g., OpenCV).

[0990] Input: Hairstyle candidate

[0991] Output: Hairstyle suggestions presented in real time

[0992] Step 5:

[0993] The user selects the hairstyle they like best from the multiple hairstyle options presented, and the selection is sent back to the server from the device.

[0994] Input: User selection

[0995] Output: The selection results are sent to the server

[0996] Step 6:

[0997] The server notifies the technician of the user's selection. The technician uses this information to prepare for the treatment. Furthermore, a prompt message based on the user's selection is sent to the technician. This allows the technician to respond quickly to the user's request.

[0998] Input: User selection

[0999] Output: Generates and sends notifications and prompts to technicians

[1000] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1001] The system of the present invention provides personalized hairstyle suggestions by combining AI technology and an emotion engine to improve the user experience in a beauty salon. This system operates in cooperation with multiple means described below.

[1002] 1. Enter and submit user information

[1003] Terminal

[1004] The user uses a dedicated terminal at the salon to input their personal information and real-time facial image. The user captures the facial image using the terminal's camera, and the user's hair type (e.g., straight or curly), bone structure, preferred hairstyle and color, and the emotion engine reads the user's emotions from their facial expressions. All of this information is encrypted and sent to the server.

[1005] 2. Analysis and style candidate generation

[1006] server

[1007] The server receives the user's facial image, personal information, and emotional data sent from the device. Based on this data, an AI model is used to extract feature data such as facial shape, hair type, skin condition, and emotional data obtained by an emotion engine, and based on this, optimal hairstyle and color candidates are generated. Multiple style candidates are generated and superimposed on the user's facial image.

[1008] 3. Presentation and selection of style candidates

[1009] Terminal

[1010] The generated style candidates are sent from the server to the device, which then presents the received multiple style candidates to the user in real time, allowing the user to check the style applied to their own face.

[1011] user

[1012] Users can choose the style they like best from a selection of suggested styles. In some cases, they can refer to detailed information and comments to select the appropriate style. Additionally, the system analyzes users' emotions in real time and reflects this information in the style selection.

[1013] 4. Notification of results and feedback

[1014] Terminal

[1015] The selection results are sent to the hairdresser in real time, who then reviews the style selected by the user and begins preparations for the treatment.

[1016] hairdresser

[1017] Based on the user's selections and emotional data, the hairdresser will hold a detailed consultation with the user and provide feedback as needed.

[1018] Specific examples

[1019] As a specific example, suppose User B visits a hair salon, captures an image of their face on their device, and enters their hair type as "curly" and their favorite color as "blonde." The device sends this information, along with the user's emotional data, to the server. The server uses an AI model and an emotion engine to generate multiple blonde hairstyles, which are then superimposed on User B's facial image and sent back to the device. User B reviews the style options presented in real time and selects the most suitable one. The emotional data is also reflected, and the user's selected style is notified to the hairdresser. The hairdresser reviews the selection and emotional data, prepares for the treatment, consults with User B, and makes any final adjustments before beginning the treatment.

[1020] In this way, by incorporating the user's emotional data, this system is able to provide more personalized style suggestions, improve user satisfaction, and enhance communication with the hairdresser.

[1021] The processing flow will be explained below.

[1022] Step 1:

[1023] The user operates the device, captures a facial image using the camera, and inputs personal information such as hair type, preferred color, and hairstyle. The emotion engine then analyzes the user's facial expressions and collects emotional data.

[1024] Step 2:

[1025] The device transmits the captured facial image, input personal information, and acquired emotion data to a server, where the data is encrypted to protect privacy.

[1026] Step 3:

[1027] The server analyzes the received facial image data, personal information, and emotional data, and extracts feature data based on the user's facial shape, hair type, skin condition, and emotional data using image analysis and emotional analysis algorithms.

[1028] Step 4:

[1029] The server inputs the extracted feature data and emotion data into an AI model to generate multiple personalized hairstyle and color suggestions based on the user's unique information and current emotion. Emotion data is given special weight when generating style suggestions.

[1030] Step 5:

[1031] The server sends the generated style candidate images to the device in real time, and the device displays the received style candidates superimposed on the user's facial image, allowing the user to check them in real time.

[1032] Step 6:

[1033] The device presents style candidates to the user, who then sees the style applied to their face. The proposed style takes into account the user's current emotional state, making it highly appropriate.

[1034] Step 7:

[1035] The user selects the style they like best from the multiple style options presented, and the selection reflects the user's emotional state, increasing satisfaction.

[1036] Step 8:

[1037] The device transmits the user's selected style results, including emotional data, to the server in real time.

[1038] Step 9:

[1039] The server processes the selection results and emotion data and sends a notification to the hairdresser's device, where the hairdresser can check the selection results and emotion data in real time.

[1040] Step 10:

[1041] The hairstylist will then hold an in-depth consultation with the user based on their selection and emotional data, providing feedback that takes their emotions into account and suggesting tweaks to the style if necessary.

[1042] Step 11:

[1043] Based on the final feedback, the hairdresser will begin the actual treatment and achieve the hairstyle that matches the user's wishes. Emotional data is taken into consideration throughout the treatment, improving the user experience.

[1044] In this way, by adding emotional data, this system can provide more personalized style suggestions, improve user satisfaction, and enhance communication with hairdressers.

[1045] Example 2

[1046] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1047] To improve customer satisfaction at hair salons, personalized style suggestions that take into account the individual characteristics and emotions of each user are required. However, conventional systems only consider basic information such as the user's facial image, hair type, and preferred style, and are unable to suggest styles that reflect detailed personal data such as the user's emotions and skin condition. This results in issues such as not being able to meet user expectations and not being able to communicate effectively with the hairdresser.

[1048] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1049] In this invention, the server includes a means for acquiring the user's emotional data and reflecting it in the creation of a style, a means for proposing a style and color based on the user's skin condition, and a means for the hairdresser to provide feedback on the result of the user's selection from the presented style candidates. This makes it possible to propose a style based on detailed personal data such as the user's facial image, hair type, and preferred style, as well as their emotions and skin condition.

[1050] A "facial image" is a digital image that captures the visual features of a user's face.

[1051] "Personal information" refers to information about the user, such as hair type, preferred hairstyle and color.

[1052] "Feature data" refers to characteristic data such as facial shape, hair type, and skin condition extracted from the user's facial image and personal information.

[1053] "Style candidates" are multiple hairstyle and color suggestions generated based on feature data, and are presented as a composite image superimposed on the user's facial image.

[1054] An "emotion engine" is software or hardware that reads emotions from a user's facial expressions and analyzes them as data.

[1055] "Real-time" refers to a state in which data and processing occur with almost no delay, allowing users and systems to respond immediately.

[1056] "Feedback" refers to any comments or information provided by a Hairdresser regarding improvements or evaluation of a User's selection.

[1057] "Suggestions" are specific style and color options and ideas to present to users.

[1058] The system of the present invention provides personalized hairstyle suggestions by combining AI technology and an emotion engine to improve the user experience in a beauty salon. This system utilizes the user's facial image, personal information, and emotion data, and includes a series of processes to generate and present optimal style suggestions in real time.

[1059] First, the user uses a dedicated terminal installed in the salon to input their individual personal information and a real-time facial image. The terminal is equipped with a camera, and the user presses the "take a photo" button to capture a facial image. Along with this facial image, the user also inputs their hair type, preferred hairstyle and color. In addition, the emotion engine installed in the terminal reads emotions from the user's facial expressions, and this information is also collected. This data is encrypted and sent to the server.

[1060] The server analyzes the received facial image, personal information, and emotional data. First, it uses a facial recognition algorithm to extract the shape and features of the face. It then analyzes the acquired emotional data using an emotion engine to recognize specific emotions. Based on this analyzed data, the server uses an AI model to generate optimal hairstyle and color candidates. Multiple style candidates are generated and superimposed on the user's facial image.

[1061] The generated style candidates are then sent from the server to the device. The device then presents the received style candidates to the user in real time. The user can then select the style they like best from the multiple style candidates presented. When making a selection, the user can refer to detailed information and comments for each style. In addition, the user's emotions are analyzed in real time, and this information is reflected in the style selection.

[1062] Once the user selects the best style, the results are communicated to the hairdresser in real time. The hairdresser then conducts a detailed consultation with the user based on the selected style and emotional data, providing feedback as needed. The hairdresser then begins preparations for the treatment, making any final adjustments before styling the user's hair.

[1063] As a concrete example, suppose User B visits a hair salon, captures a facial image on a device, and inputs "hair type: curly" and "preferred color: blonde." This data and emotional data are sent to the server. The server uses an AI model and an emotion engine to generate a blonde style for User B and overlays it on the user's facial image. The generated style candidates are displayed on the device, and User B selects the most suitable style. The selection result, reflecting User B's emotions, is notified to the hairdresser, who makes preparations based on that information and begins the treatment after a final consultation.

[1064] An example prompt is:

[1065] "I'm a woman in her 20s with a cheerful face and curly hair. My favorite color is blonde. Please generate the best hairstyle for her."

[1066] By incorporating user emotional data, the system can provide personalized style suggestions, improve user satisfaction, and enhance communication with hairdressers.

[1067] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1068] Step 1: Enter your user information

[1069] Terminal

[1070] The user uses a dedicated terminal at the salon and first enters their personal information. The terminal is equipped with a camera, and the user captures a facial image by facing the camera and pressing the "shoot" button. Specifically, the facial image (input) is obtained and the data is temporarily saved (output). The user also inputs their hair type (e.g. straight hair, curly hair), preferred hairstyle and color (input) through the UI. Along with this information, the emotion engine analyzes facial expressions and obtains emotional data (output).

[1071] Step 2: Submit user information

[1072] Terminal

[1073] The captured facial image, input hair type, preferred hairstyle and color, and acquired emotion data are all encrypted. The encrypted data packet is sent to the server. The input is the entire user information, and the output is the encrypted data packet.

[1074] Step 3: Receiving and analyzing user information

[1075] server

[1076] The server receives encrypted data packets from the device (input). The received data is decrypted and separated into facial images, personal information, and emotional data (output). The facial images are then analyzed using a facial recognition algorithm to extract facial shapes and features, generating feature data (output). The emotion engine then analyzes the acquired emotional data to identify specific emotions (e.g., joy, surprise, sadness).

[1077] Step 4: Generate style candidates

[1078] server

[1079] The server operates a generative AI model based on the analyzed feature data and emotional data (input). This AI model combines the user's face shape, hair type, personal information, and emotional data to generate multiple candidates for optimal hairstyles and colors (output). It also creates a composite image in which these styles are superimposed on the user's face image. Style candidate data including this composite image is generated.

[1080] Step 5: Submit and present your style suggestions

[1081] server

[1082] The generated style candidates and the composite image are sent to the terminal (the input is the style candidate data, and the output is the sent data). This is also encrypted and transmitted securely.

[1083] Terminal

[1084] The device decodes the received style candidate data (input) and presents multiple style candidates and their detailed information to the user in real time (output). The user can check each style through the UI. For example, pressing the "Next" button displays the next style candidate.

[1085] Step 6: User Style Selection

[1086] user

[1087] The user selects the style they like best from the multiple style candidates presented (input: user selection). By pressing the "Select" button on the UI, that style is selected (output). If necessary, detailed information and comments for each style can be viewed to help with the final decision. Additionally, the emotion engine analyzes the user's facial expressions while selecting and reflects them in the selection results.

[1088] Step 7: Notification of results and feedback

[1089] Terminal

[1090] The server notifies the user of the style they have selected in real time (the input is the selection result, and the output is the notification data). This information is then forwarded to the hairdresser's device.

[1091] hairdresser

[1092] The hairdresser confirms the user's selection and emotional data (input) and begins preparations for the treatment. He / she then consults with the user (output) and provides necessary feedback, such as explaining the feasibility of the selected style and the necessary treatments. After making any final adjustments, the treatment begins.

[1093] Through this series of processes, personalized style suggestions and high-quality treatments are realized based on the user's emotions and personal information.

[1094] (Application example 2)

[1095] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1096] Hair salons need to propose optimal hairstyles based on the individual feelings and preferences of their customers, thereby increasing customer satisfaction. However, conventional systems have the problem that customers cannot confirm which style suits them until they actually visit the salon. Furthermore, there is no way for users to try the style at home before the treatment, and due to a lack of communication with the hairdresser, the desired style may not be fully reflected.

[1097] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1098] In this invention, the server includes: means for inputting a user's facial image and personal information; means for analyzing the facial image and personal information and extracting feature data; means for generating multiple personalized style candidates based on the extracted feature data; means for presenting the generated style candidates to the user in real time; means for notifying the hairdresser of the user's selected style; means for the user to use the virtual beauty salon app at home; means for capturing a facial image and inputting a desired hairstyle using a smartphone; and means for generating a hairstyle in real time using an AI model and an emotion engine through the virtual beauty salon app. This allows the user to try out multiple hairstyles at home and select the best style before visiting the salon. As a result, user satisfaction is improved and communication with the hairdresser is smoother.

[1099] "User" refers to an individual who uses the services of a beauty salon.

[1100] "Facial image" means a photograph or image data of a user's face.

[1101] "Personal Information" refers to your hair type, preferred hairstyles and colors, and other personal information.

[1102] "Feature data" refers to data such as facial shape, hair type, and skin condition analyzed from the input facial image and personal information.

[1103] "Style suggestions" refers to multiple hairstyles suggested to the user.

[1104] "Real-time" means immediate response or processing without delay.

[1105] "Practitioner" refers to a professional who performs hairstyles in a beauty salon, i.e., a hairdresser.

[1106] A "virtual beauty salon app" refers to application software that allows users to try out hairstyles at home using their smartphones.

[1107] "Smartphone" means a portable communication device that has Internet connectivity and can be used by installing applications.

[1108] An "AI model" refers to artificial intelligence that uses technologies such as machine learning and deep learning to analyze a user's characteristic data and generate optimal style candidates.

[1109] "Emotion engine" refers to technology that reads emotions from a user's facial expressions and uses those emotions as data for analysis.

[1110] The system of the present invention provides personalized hairstyle suggestions that combine AI technology and an emotion engine to improve the user experience in beauty salons. This system includes a means for inputting a user's facial image and personal information, a means for analyzing the facial image and personal information and extracting feature data, a means for generating multiple personalized style candidates based on the extracted feature data, a means for presenting the generated style candidates to the user in real time, and a means for notifying the hairdresser of the style results selected by the user.

[1111] 1. Enter and submit user information

[1112] Terminal

[1113] Users use their smartphone to capture a photo of their face at home or at a dedicated terminal in a beauty salon, and input their hair type, preferred hairstyle, and color. The facial image is then captured using the smartphone's camera or the dedicated terminal's camera. This data is encrypted as needed and sent to a server.

[1114] 2. Analysis and style candidate generation

[1115] server

[1116] After receiving the user's personal information and facial image, the server uses an AI model to analyze facial shape, hair type, skin condition, and even emotional data obtained from an emotion engine to extract feature data. Multiple style candidates are then generated based on the extracted feature data. The generated style candidates are then overlaid on the user's facial image to present a realistic image.

[1117] 3. Presentation and selection of style candidates

[1118] Terminal

[1119] The generated style candidates are sent from the server to the device and are presented to the user in real time on a smartphone or dedicated device. The user can then review multiple style candidates applied to their face and select the one that best suits them.

[1120] user

[1121] Users can select their favorite style from the suggested styles, and the system will also incorporate suggestions based on emotional data to optimize the selection.

[1122] 4. Notification of results and feedback

[1123] Terminal

[1124] The results of the selected style are notified to the practitioner in real time, who then confirms the style selected by the user and begins preparations for the treatment.

[1125] Practitioner

[1126] The practitioner will conduct a detailed consultation based on the user's selections, provide feedback as needed, and make any final adjustments before beginning the treatment.

[1127] Hardware and software used

[1128] In this invention, in addition to the camera of a smartphone or dedicated device, a server is used to run the AI ​​model, and existing AI services such as Azure Face API, Google Cloud Vision, and Amazon Rekognition are used for the analysis.

[1129] Specific examples

[1130] For example, when User A uses this system at home, they capture a photo of their face using their smartphone and enter their hair type as "curly" and color as "red." The smartphone encrypts this information and sends it to the server. The server-side AI model analyzes this data, generates multiple personalized style options for User A, and sends the results to the smartphone. Once User A selects the optimal style, the results are notified to the hairdresser in real time.

[1131] Prompt Sentence Examples

[1132] "Generate the best hairstyle candidates for the user based on their face image and preferred hairstyle information. The hair type is "curly hair", the hair color is "red", and the specific face image is attached below."

[1133] The above is a detailed description of the embodiment of the present invention. This system allows users to try out multiple hairstyles at home and choose the best one before visiting a beauty salon. As a result, user satisfaction is improved and communication with the hairdresser is smoother.

[1134] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1135] Step 1:

[1136] Users use a smartphone or dedicated device to capture an image of their face and enter personal information.

[1137] Specific operation: The user activates the smartphone camera and takes a picture of their face, and also inputs their hair type (e.g., curly hair), preferred hairstyle, and color (e.g., red).

[1138] Input: User's face image, hair type, preferred hairstyle and color

[1139] Output: Captured face image and input personal information

[1140] Step 2:

[1141] The facial image and personal information acquired by the terminal are transmitted to the server.

[1142] Specific operation: Using the device's built-in transmission function, the encrypted facial image and personal information are uploaded to the server.

[1143] Input: Captured face image and input personal information

[1144] Output: Facial image and personal information sent to the server

[1145] Step 3:

[1146] The server analyzes the received facial image and personal information.

[1147] How it works: Based on the data received by the server, it uses an AI model and emotion engine to extract facial shape, hair type, skin condition, and emotion data.

[1148] Input: Submitted face image and personal information

[1149] Output: Analyzed feature data (face shape, hair type, skin condition, emotion data)

[1150] Step 4:

[1151] The server generates multiple personalized style candidates based on the extracted feature data.

[1152] How it works: Using an AI model, it generates multiple hairstyles that best fit the user's feature data.

[1153] Input: Parsed feature data

[1154] Output: Image data of style candidates

[1155] Step 5:

[1156] The generated style candidates are sent to the device and presented to the user in real time.

[1157] Specific operation: The server sends image data of the generated style candidates to the terminal, which then displays them on the screen.

[1158] Input: Image data of generated style candidates

[1159] Output: Style suggestions presented to the user

[1160] Step 6:

[1161] The user selects their favorite style from the suggested styles.

[1162] What it does: The user selects the best style from the displayed options and confirms the selection.

[1163] Input: suggested style suggestions

[1164] Output: User selected style

[1165] Step 7:

[1166] The terminal notifies the practitioner of the style results selected by the user.

[1167] Specific operation: The selected style results are sent to the practitioner using the device's built-in notification function.

[1168] Input: User selected style

[1169] Output: Style results communicated to the practitioner

[1170] Step 8:

[1171] The therapist checks the selected style results and the user's emotional data, and begins preparations for the treatment.

[1172] Specific actions: The practitioner prepares for the actual treatment based on the notified style results, and also conducts detailed consultations with the user and provides feedback as needed.

[1173] Input: Notified style results and emotion data

[1174] Output: Prepared treatment and provided feedback

[1175] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1176] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1177] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[1178] [Fourth embodiment]

[1179] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[1180] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1181] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1182] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[1183] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1184] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1185] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1186] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[1187] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1188] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1189] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1190] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1191] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1192] The system of the present invention provides personalized hairstyle suggestions using AI technology to improve the user experience at a beauty salon. This system operates in cooperation with multiple means described below.

[1193] 1. Enter and submit user information

[1194] Terminal

[1195] Users use dedicated terminals in beauty salons to input their individual personal information and real-time facial images. The user captures a facial image using the terminal's camera and inputs information about hair type (e.g., straight or curly), bone structure, and preferred hairstyle and color. All this information is encrypted and sent to a server for further processing.

[1196] 2. Analysis and style candidate generation

[1197] server

[1198] The server receives the user's facial image and personal information sent from the device. Based on this data, an AI model is used to extract characteristic data such as facial shape, hair type, and skin condition, and based on that, optimal hairstyle and color candidates are generated. Multiple style candidates are generated and superimposed on the user's facial image.

[1199] 3. Presentation and selection of style candidates

[1200] Terminal

[1201] The generated style candidates are sent from the server to the device, which then presents the received multiple style candidates to the user in real time, allowing the user to check how the style will look on their actual appearance.

[1202] user

[1203] Users are presented with multiple style options and can choose the one they like best. In some cases, they can also refer to detailed information and comments to help them choose the appropriate style.

[1204] 4. Notification of results and feedback

[1205] Terminal

[1206] The selection results are sent to the hairdresser in real time, who then reviews the style selected by the user and begins preparations for the treatment.

[1207] hairdresser

[1208] Based on the user's chosen style, the stylist will hold a detailed consultation with the user before carrying out any necessary treatments. The stylist will provide feedback on the user's chosen style and make any final adjustments.

[1209] Specific examples

[1210] As a specific example, suppose User A visits a hair salon, captures an image of their face on their device, and enters their hair type as "straight" and their favorite color as "brown." The device sends this information to the server, which uses an AI model to generate multiple brown hairstyles. These styles are sent back to the device superimposed on User A's face image, allowing User A to view them in real time. User A then selects their favorite style, and the result is notified to the hairdresser. After reviewing and consulting with User A, the hairdresser begins treatment based on the selected style.

[1211] In this way, the system enhances communication between users and hairdressers, providing optimal styles tailored to individual needs and improving the salon experience.

[1212] The processing flow will be explained below.

[1213] Step 1:

[1214] The user operates the device, captures a facial image using the camera, and inputs personal information such as hair type, preferred color, and hairstyle.

[1215] Step 2:

[1216] The device sends the captured facial image and input personal information to a server, where the data is encrypted to protect privacy.

[1217] Step 3:

[1218] The server analyzes the received facial image data and personal information, and extracts characteristic data such as the user's facial shape and hair type using an image analysis algorithm.

[1219] Step 4:

[1220] The server inputs the extracted feature data into an AI model to generate multiple personalized hairstyle and color suggestions based on the user's specific information.

[1221] Step 5:

[1222] The server then sends the generated style candidate images to the device in real time, which then displays an image of the user's face with the style applied.

[1223] Step 6:

[1224] The device then presents the received style candidate images to the user, who can then view the style applied to their own face on the device screen.

[1225] Step 7:

[1226] Users can select the style they like best from the suggested styles, and can also view detailed information and comments about the selected style.

[1227] Step 8:

[1228] The device sends the user's selected style results to the server in real time.

[1229] Step 9:

[1230] The server processes the selection results and sends a notification to the hairdresser's device, where the hairdresser can see the user's selection results in real time.

[1231] Step 10:

[1232] After reviewing the user's chosen style, the hairstylist will hold a detailed consultation with the user and provide feedback as needed.

[1233] Step 11:

[1234] Based on the final feedback, the hairdresser will begin the actual treatment and create a hairstyle that meets the customer's wishes.

[1235] Example 1

[1236] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1237] Currently, hair salons have difficulty proposing hairstyles that effectively reflect the individual preferences and characteristics of their customers. Furthermore, there is a lack of means for customers to quickly communicate their chosen style to the hairdresser and incorporate their feedback. As a result, achieving the hairstyle the customer desires often requires time and effort.

[1238] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1239] In this invention, the server includes means for inputting a user's facial image and personal information, means for encrypting the facial image and personal information and transmitting them to the server, means for decrypting the encrypted data and extracting feature data in the server, means for generating a plurality of personalized style candidates using a generative AI model based on the extracted feature data, means for presenting the generated style candidates to the user in real time, and means for notifying the provider of the style results selected by the user. This enables the server to quickly propose an optimal hairstyle according to the user's individual preferences and facilitates smooth communication with the hairdresser.

[1240] A "user's facial image" is video data that includes the facial features of an individual user.

[1241] "Personal Information" refers to information about a user, such as their hair type, bone structure, and preferred styles and colors.

[1242] "Encryption" is the process of using a cryptographic standard (e.g., AES-256) to transmit data securely and converting the content into a format that is unreadable to third parties.

[1243] "Feature data" refers to user-specific data points and attributes extracted based on the user's facial image and personal information.

[1244] A "generative AI model" is a model that uses artificial intelligence technology to generate optimal hairstyle candidates from input data.

[1245] "Style suggestions" are multiple hairstyle options generated by the AI ​​model to suggest to the user.

[1246] "Real-time" refers to processing within a timeframe that provides an immediate response to user interaction.

[1247] A "provider" is a professional who provides a service to a user, such as a hairdresser.

[1248] "Feedback" is any evaluation, opinion, or advice from the provider regarding the style selected by the user.

[1249] This invention is a system for providing personalized hairstyle suggestions that reflect the individual preferences and characteristics of users at beauty salons. The system includes a series of techniques for encrypting data entered by users using a dedicated terminal, analyzing it on a server, and generating optimal style suggestions.

[1250] Hardware and software used

[1251] The server uses NVIDIA GPUs (e.g., NVIDIA RTX 3080) and TensorFlow to run AI models, and the OpenSSL library is used for data encryption and decryption. The device is equipped with a camera, which is used to capture the user's facial image.

[1252] System Operation Overview

[1253] Terminal

[1254] Users use a dedicated terminal at the salon to input their facial image and personal information (e.g., hair type, bone structure, preferred hairstyle and color). Specifically, users capture a facial image using the terminal's camera and input various personal information on the screen. All of this information is encrypted using AES-256 to ensure security. The encrypted data is then sent to a server via the Internet.

[1255] server

[1256] The server receives the encrypted data sent from the device and decrypts it using AES-256. The decrypted data includes the user's facial image and personal information. The server then runs an AI model using TensorFlow to extract the user's feature data from the facial image and personal information. Based on this feature data, the server generates multiple style candidates, including optimal hairstyles and colors. The generated style candidates are then overlaid on the user's facial image.

[1257] Terminal

[1258] The generated style candidates are sent from the server to the device. The device then presents these style candidates to the user in real time, allowing the user to see how they will look when applied to their own face. The user can then select the style candidate they like best from the multiple style candidates presented. If necessary, they can refer to detailed information and comments.

[1259] user

[1260] Users can compare multiple style options displayed in real time and select the hairstyle that best suits them. The results of this selection are then sent to the provider (e.g., hairdresser) in real time.

[1261] provider

[1262] The provider will review the user's selections on the device and begin preparing the necessary treatment. During a detailed consultation with the user, the provider will provide feedback based on the style selected and make any final adjustments.

[1263] Specific examples

[1264] For example, let's say User A visits a hair salon. First, User A uses a device to capture an image of their face, and enters their hair type as "straight" and their preferred color as "brown." This information is encrypted on the device and sent to the server. The server decrypts the encrypted data and uses an AI model to generate multiple brown hairstyle candidates. These generated style candidates are superimposed on User A's face image and sent back to the device. User A compares the displayed multiple style candidates and selects the one they like best. The selection result is notified to the salon provider, who then begins preparations for the treatment and, after a detailed consultation with User A, adjusts the style to the final version.

[1265] Prompt Sentence Examples

[1266] "Run an AI model to suggest the best hairstyle options based on the user's facial image and personal information."

[1267] This system quickly suggests the best hairstyle based on the user's individual preferences and enables smooth communication with the hairdresser, which is expected to improve the user experience at the salon and increase satisfaction.

[1268] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1269] Step 1:

[1270] The user inputs his / her facial image and personal information using a dedicated terminal at the beauty salon.

[1271] Specifically, users use the device's camera to capture a facial image and enter personal information such as hair type, bone structure, and preferred hairstyle and color.

[1272] Input: User's face image and personal information.

[1273] Output: The encrypted data.

[1274] Step 2:

[1275] The device encrypts the entered facial image and personal information using AES-256.

[1276] Specifically, the terminal encrypts input data and executes processes to maintain security.

[1277] Input: User's face image and personal information.

[1278] Output: The encrypted data.

[1279] Step 3:

[1280] The encrypted data is sent to a server over the Internet.

[1281] Specifically, the terminal executes a process of transmitting encrypted data to a server via a network.

[1282] Input: Encrypted data.

[1283] Output: The data sent to the server.

[1284] Step 4:

[1285] The server decrypts the received encrypted data using AES-256.

[1286] Specifically, the server runs a process to decrypt the encrypted data and retrieve the original facial image and personal information.

[1287] Input: Encrypted data.

[1288] Output: Decoded face image and personal information.

[1289] Step 5:

[1290] The server extracts feature data based on the decoded face image and personal information.

[1291] Specifically, the server runs an AI model to extract feature data from the user's facial image and personal information.

[1292] Input: Decoded face image and personal information.

[1293] Output: Extracted feature data.

[1294] Step 6:

[1295] The server generates multiple personalized style candidates using a generative AI model based on the extracted feature data.

[1296] Specifically, the server uses an AI model to generate multiple hairstyle options that are overlaid on the user's face image.

[1297] Input: Extracted feature data.

[1298] Output: Multiple style candidates.

[1299] Step 7:

[1300] The server sends the generated style candidates to the terminal in real time.

[1301] Specifically, a process is executed to transmit the generated style candidate data to the terminal via the Internet.

[1302] Input: Multiple style candidates.

[1303] Output: Data sent to the terminal.

[1304] Step 8:

[1305] The device presents the received style suggestions to the user in real time.

[1306] Specifically, the device executes a process to display style candidates superimposed on the user's facial image.

[1307] Input: Style suggestions received from the server.

[1308] Output: Multiple style suggestions displayed to the user.

[1309] Step 9:

[1310] The user selects the style they like best from the multiple style options presented.

[1311] Specifically, the user compares style candidates on the device screen and makes a selection.

[1312] Input: Several style suggestions displayed on the device.

[1313] Output: User selected style.

[1314] Step 10:

[1315] The device notifies the provider of the user's selected style results in real time.

[1316] Specifically, the terminal executes a process of transmitting the selection results to the provider in real time via the network.

[1317] Input: The style selected by the user.

[1318] Output: The selection results communicated to the provider.

[1319] Step 11:

[1320] The provider will review the user's selections and begin preparing the necessary treatment.

[1321] Specifically, the provider checks the selection results on the terminal and plans the treatment in consultation with the user.

[1322] Input: The selection results notified to the provider.

[1323] Output: Preparing and planning the procedure.

[1324] Step 12:

[1325] The provider will hold a detailed consultation with the user and provide feedback based on the style selected.

[1326] Specifically, the provider will actually have a conversation with the user and make final style adjustments.

[1327] Input: The selection results notified to the provider.

[1328] Output: Feedback and final adjustments provided to the user.

[1329] (Application example 1)

[1330] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1331] In modern society, the expectations and needs of users of beauty services are becoming more diverse. While remote home beauty treatments and virtual salon experiences are on the rise, choosing the hairstyle and color that best suits them can be a challenging task. Furthermore, the time and expense required to actually try on a selected style can reduce user satisfaction. Given this background, there is a demand for a system that allows users to try on various hairstyles in real time from the comfort of their own home and select the style that best suits them.

[1332] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1333] In this invention, the server includes a device for inputting a user's facial image and personal information, a device for analyzing the facial image and personal information and extracting feature data, a device for generating a plurality of personalized style candidates based on the extracted feature data, a device for presenting the generated style candidates to the user in real time, a device for notifying the technician of the style selected by the user, and a device for transmitting the selected style, presenting the style in real time, and generating a prompt message for accepting the selection. This allows the user to try out various hairstyles in real time from the comfort of their own home, select the best style, and immediately notify the technician.

[1334] "Device for inputting a user's facial image and personal information" refers to a terminal or application that allows a user to input their own facial photo and personal information (e.g., hair type, preferred color, etc.).

[1335] "Device that analyzes facial images and personal information and extracts feature data" refers to a server or system that processes input facial images and personal information and uses AI technology to analyze feature data such as facial shape and hair type.

[1336] A "device for generating multiple personalized style candidates" is a system that includes an algorithm or AI model that automatically generates optimal style and hair color candidates for a user based on extracted feature data.

[1337] The "device that presents generated style candidates to the user in real time" refers to a terminal or application that displays a plurality of generated style candidates to the user in real time for confirmation.

[1338] The "device for notifying a technician of the style results selected by a user" refers to a system for notifying a technician (e.g., a hairdresser or hair stylist) of the style information selected by a user in real time.

[1339] "A device that transmits selected styles, presents styles in real time, and generates prompts to accept selections" refers to a device that transmits style information and provides an interface for selection using prompts that are generated based on user selections.

[1340] The system that realizes this application example is composed of a device that inputs a user's facial image and personal information, a server that analyzes the facial image and personal information and extracts feature data, a server that generates personalized style candidates based on the extracted feature data, a terminal that presents the generated style candidates to the user in real time, a device that notifies the engineer of the selected style result, and a device that transmits the selected style and generates a prompt sentence.

[1341] Specific operation of the system

[1342] The operation of this system is as follows.

[1343] 1. Enter and submit user information

[1344] Users use a dedicated device or smartphone app to capture an image of their face and enter personal information such as hair type and preferred color. The data is encrypted and then sent to a server. Logitec webcams and other cameras are used as camera devices, and the Python OpenCV library is used for image processing.

[1345] 2. Analysis and style candidate generation

[1346] The submitted facial image and personal information are received by a server and analyzed using an AI model. The server extracts feature data such as facial shape and hair type, and generates multiple hairstyle candidates based on that data. This AI model is built using deep learning frameworks such as TensorFlow and Keras.

[1347] 3. Presentation and selection of style candidates

[1348] The generated hairstyle candidates are sent from the server to the user's device. The user can view the hairstyles in real time and select the style they like best. This process uses OpenCV image display technology. The selected style is then sent back to the server.

[1349] 4. Notification of results and feedback

[1350] The selection results are notified to the technician in real time, and the technician can use that information to prepare for the procedure. The technician can provide feedback to the user and make final adjustments. A prompt generation device is then used to generate a prompt based on the user's selection and send it to the technician.

[1351] Examples of prompt statements

[1352] A specific example of a prompt is as follows:

[1353] Based on the user's face image and the following information, please provide five suitable hairstyle options.

[1354] Hair Type: Straight

[1355] Favorite color: Brown

[1356] Facial images are provided in the following formats:

[1357] <Image data>

[1358] In this way, users can virtually try out different styles from the comfort of their own home, with the results communicated to the technician in real time, which is expected to provide a smooth and efficient beauty experience.

[1359] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1360] Step 1:

[1361] Users use a dedicated device or smartphone app to capture an image of their face and enter personal information such as hair type and preferred color. The facial image is captured using a camera device (e.g., a Logitec webcam). The personal information is collected in text format based on the user's input. This data is encrypted and then sent to the server.

[1362] Input: Personal information such as the user's face image, hair type, favorite color, etc.

[1363] Output: Encrypted face image and personal information are sent to the server

[1364] Step 2:

[1365] The server analyzes the received facial image and personal information and extracts feature data from them. Specifically, it uses image analysis technology (e.g., OpenCV) to detect facial shape, hair type, skin condition, etc. It also analyzes the input personal information to extract information about preferred styles.

[1366] Input: Encrypted face image and personal information

[1367] Output: Feature data such as face shape, hair texture, skin condition, etc.

[1368] Step 3:

[1369] The server generates personalized hairstyle candidates based on the extracted feature data. To do this, it uses a generative AI model (e.g., a deep learning model using TensorFlow or Keras). This model generates multiple hairstyles that match the user's features.

[1370] Input: feature data

[1371] Output: Multiple personalized hairstyle suggestions

[1372] Step 4:

[1373] The server sends the generated hairstyle candidates to the user's device, which then presents them to the user in real time. Each proposed style is combined with the facial image, and the user can view the style displayed through the interface. This process uses image display technology (e.g., OpenCV).

[1374] Input: Hairstyle candidate

[1375] Output: Hairstyle suggestions presented in real time

[1376] Step 5:

[1377] The user selects the hairstyle they like best from the multiple hairstyle options presented, and the selection is sent back to the server from the device.

[1378] Input: User selection

[1379] Output: The selection results are sent to the server

[1380] Step 6:

[1381] The server notifies the technician of the user's selection. The technician uses this information to prepare for the treatment. Furthermore, a prompt message based on the user's selection is sent to the technician. This allows the technician to respond quickly to the user's request.

[1382] Input: User selection

[1383] Output: Generates and sends notifications and prompts to technicians

[1384] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1385] The system of the present invention provides personalized hairstyle suggestions by combining AI technology and an emotion engine to improve the user experience in a beauty salon. This system operates in cooperation with multiple means described below.

[1386] 1. Enter and submit user information

[1387] Terminal

[1388] The user uses a dedicated terminal at the salon to input their personal information and real-time facial image. The user captures the facial image using the terminal's camera, and the user's hair type (e.g., straight or curly), bone structure, preferred hairstyle and color, and the emotion engine reads the user's emotions from their facial expressions. All of this information is encrypted and sent to the server.

[1389] 2. Analysis and style candidate generation

[1390] server

[1391] The server receives the user's facial image, personal information, and emotional data sent from the device. Based on this data, an AI model is used to extract feature data such as facial shape, hair type, skin condition, and emotional data obtained by an emotion engine, and based on this, optimal hairstyle and color candidates are generated. Multiple style candidates are generated and superimposed on the user's facial image.

[1392] 3. Presentation and selection of style candidates

[1393] Terminal

[1394] The generated style candidates are sent from the server to the device, which then presents the received multiple style candidates to the user in real time, allowing the user to check the style applied to their own face.

[1395] user

[1396] Users can choose the style they like best from a selection of suggested styles. In some cases, they can refer to detailed information and comments to select the appropriate style. Additionally, the system analyzes users' emotions in real time and reflects this information in the style selection.

[1397] 4. Notification of results and feedback

[1398] Terminal

[1399] The selection results are sent to the hairdresser in real time, who then reviews the style selected by the user and begins preparations for the treatment.

[1400] hairdresser

[1401] Based on the user's selections and emotional data, the hairdresser will hold a detailed consultation with the user and provide feedback as needed.

[1402] Specific examples

[1403] As a specific example, suppose User B visits a hair salon, captures an image of their face on their device, and enters their hair type as "curly" and their favorite color as "blonde." The device sends this information, along with the user's emotional data, to the server. The server uses an AI model and an emotion engine to generate multiple blonde hairstyles, which are then superimposed on User B's facial image and sent back to the device. User B reviews the style options presented in real time and selects the most suitable one. The emotional data is also reflected, and the user's selected style is notified to the hairdresser. The hairdresser reviews the selection and emotional data, prepares for the treatment, consults with User B, and makes any final adjustments before beginning the treatment.

[1404] In this way, by incorporating the user's emotional data, this system is able to provide more personalized style suggestions, improve user satisfaction, and enhance communication with the hairdresser.

[1405] The processing flow will be explained below.

[1406] Step 1:

[1407] The user operates the device, captures a facial image using the camera, and inputs personal information such as hair type, preferred color, and hairstyle. The emotion engine then analyzes the user's facial expressions and collects emotional data.

[1408] Step 2:

[1409] The device transmits the captured facial image, input personal information, and acquired emotion data to a server, where the data is encrypted to protect privacy.

[1410] Step 3:

[1411] The server analyzes the received facial image data, personal information, and emotional data, and extracts feature data based on the user's facial shape, hair type, skin condition, and emotional data using image analysis and emotional analysis algorithms.

[1412] Step 4:

[1413] The server inputs the extracted feature data and emotion data into an AI model to generate multiple personalized hairstyle and color suggestions based on the user's unique information and current emotion. Emotion data is given special weight when generating style suggestions.

[1414] Step 5:

[1415] The server sends the generated style candidate images to the device in real time, and the device displays the received style candidates superimposed on the user's facial image, allowing the user to check them in real time.

[1416] Step 6:

[1417] The device presents style candidates to the user, who then sees the style applied to their face. The proposed style takes into account the user's current emotional state, making it highly appropriate.

[1418] Step 7:

[1419] The user selects the style they like best from the multiple style options presented, and the selection reflects the user's emotional state, increasing satisfaction.

[1420] Step 8:

[1421] The device transmits the user's selected style results, including emotional data, to the server in real time.

[1422] Step 9:

[1423] The server processes the selection results and emotion data and sends a notification to the hairdresser's device, where the hairdresser can check the selection results and emotion data in real time.

[1424] Step 10:

[1425] The hairstylist will then hold an in-depth consultation with the user based on their selection and emotional data, providing feedback that takes their emotions into account and suggesting tweaks to the style if necessary.

[1426] Step 11:

[1427] Based on the final feedback, the hairdresser will begin the actual treatment and achieve the hairstyle that matches the user's wishes. Emotional data is taken into consideration throughout the treatment, improving the user experience.

[1428] In this way, by adding emotional data, this system can provide more personalized style suggestions, improve user satisfaction, and enhance communication with hairdressers.

[1429] Example 2

[1430] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1431] To improve customer satisfaction at hair salons, personalized style suggestions that take into account the individual characteristics and emotions of each user are required. However, conventional systems only consider basic information such as the user's facial image, hair type, and preferred style, and are unable to suggest styles that reflect detailed personal data such as the user's emotions and skin condition. This results in issues such as not being able to meet user expectations and not being able to communicate effectively with the hairdresser.

[1432] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1433] In this invention, the server includes a means for acquiring the user's emotional data and reflecting it in the creation of a style, a means for proposing a style and color based on the user's skin condition, and a means for the hairdresser to provide feedback on the result of the user's selection from the presented style candidates. This makes it possible to propose a style based on detailed personal data such as the user's facial image, hair type, and preferred style, as well as their emotions and skin condition.

[1434] A "facial image" is a digital image that captures the visual features of a user's face.

[1435] "Personal information" refers to information about the user, such as hair type, preferred hairstyle and color.

[1436] "Feature data" refers to characteristic data such as facial shape, hair type, and skin condition extracted from the user's facial image and personal information.

[1437] "Style candidates" are multiple hairstyle and color suggestions generated based on feature data, and are presented as a composite image superimposed on the user's facial image.

[1438] An "emotion engine" is software or hardware that reads emotions from a user's facial expressions and analyzes them as data.

[1439] "Real-time" refers to a state in which data and processing occur with almost no delay, allowing users and systems to respond immediately.

[1440] "Feedback" refers to any comments or information provided by a Hairdresser regarding improvements or evaluation of a User's selection.

[1441] "Suggestions" are specific style and color options and ideas to present to users.

[1442] The system of the present invention provides personalized hairstyle suggestions by combining AI technology and an emotion engine to improve the user experience in a beauty salon. This system utilizes the user's facial image, personal information, and emotion data, and includes a series of processes to generate and present optimal style suggestions in real time.

[1443] First, the user uses a dedicated terminal installed in the salon to input their individual personal information and a real-time facial image. The terminal is equipped with a camera, and the user presses the "take a photo" button to capture a facial image. Along with this facial image, the user also inputs their hair type, preferred hairstyle and color. In addition, the emotion engine installed in the terminal reads emotions from the user's facial expressions, and this information is also collected. This data is encrypted and sent to the server.

[1444] The server analyzes the received facial image, personal information, and emotional data. First, it uses a facial recognition algorithm to extract the shape and features of the face. It then analyzes the acquired emotional data using an emotion engine to recognize specific emotions. Based on this analyzed data, the server uses an AI model to generate optimal hairstyle and color candidates. Multiple style candidates are generated and superimposed on the user's facial image.

[1445] The generated style candidates are then sent from the server to the device. The device then presents the received style candidates to the user in real time. The user can then select the style they like best from the multiple style candidates presented. When making a selection, the user can refer to detailed information and comments for each style. In addition, the user's emotions are analyzed in real time, and this information is reflected in the style selection.

[1446] Once the user selects the best style, the results are communicated to the hairdresser in real time. The hairdresser then conducts a detailed consultation with the user based on the selected style and emotional data, providing feedback as needed. The hairdresser then begins preparations for the treatment, making any final adjustments before styling the user's hair.

[1447] As a concrete example, suppose User B visits a hair salon, captures a facial image on a device, and inputs "hair type: curly" and "preferred color: blonde." This data and emotional data are sent to the server. The server uses an AI model and an emotion engine to generate a blonde style for User B and overlays it on the user's facial image. The generated style candidates are displayed on the device, and User B selects the most suitable style. The selection result, reflecting User B's emotions, is notified to the hairdresser, who makes preparations based on that information and begins the treatment after a final consultation.

[1448] An example prompt is:

[1449] "I'm a woman in her 20s with a cheerful face and curly hair. My favorite color is blonde. Please generate the best hairstyle for her."

[1450] By incorporating user emotional data, the system can provide personalized style suggestions, improve user satisfaction, and enhance communication with hairdressers.

[1451] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1452] Step 1: Enter your user information

[1453] Terminal

[1454] The user uses a dedicated terminal at the salon and first enters their personal information. The terminal is equipped with a camera, and the user captures a facial image by facing the camera and pressing the "shoot" button. Specifically, the facial image (input) is obtained and the data is temporarily saved (output). The user also inputs their hair type (e.g. straight hair, curly hair), preferred hairstyle and color (input) through the UI. Along with this information, the emotion engine analyzes facial expressions and obtains emotional data (output).

[1455] Step 2: Submit user information

[1456] Terminal

[1457] The captured facial image, input hair type, preferred hairstyle and color, and acquired emotion data are all encrypted. The encrypted data packet is sent to the server. The input is the entire user information, and the output is the encrypted data packet.

[1458] Step 3: Receiving and analyzing user information

[1459] server

[1460] The server receives encrypted data packets from the device (input). The received data is decrypted and separated into facial images, personal information, and emotional data (output). The facial images are then analyzed using a facial recognition algorithm to extract facial shapes and features, generating feature data (output). The emotion engine then analyzes the acquired emotional data to identify specific emotions (e.g., joy, surprise, sadness).

[1461] Step 4: Generate style candidates

[1462] server

[1463] The server operates a generative AI model based on the analyzed feature data and emotional data (input). This AI model combines the user's face shape, hair type, personal information, and emotional data to generate multiple candidates for optimal hairstyles and colors (output). It also creates a composite image in which these styles are superimposed on the user's face image. Style candidate data including this composite image is generated.

[1464] Step 5: Submit and present your style suggestions

[1465] server

[1466] The generated style candidates and the composite image are sent to the terminal (the input is the style candidate data, and the output is the sent data). This is also encrypted and transmitted securely.

[1467] Terminal

[1468] The device decodes the received style candidate data (input) and presents multiple style candidates and their detailed information to the user in real time (output). The user can check each style through the UI. For example, pressing the "Next" button displays the next style candidate.

[1469] Step 6: User Style Selection

[1470] user

[1471] The user selects the style they like best from the multiple style candidates presented (input: user selection). By pressing the "Select" button on the UI, that style is selected (output). If necessary, detailed information and comments for each style can be viewed to help with the final decision. Additionally, the emotion engine analyzes the user's facial expressions while selecting and reflects them in the selection results.

[1472] Step 7: Notification of results and feedback

[1473] Terminal

[1474] The server notifies the user of the style they have selected in real time (the input is the selection result, and the output is the notification data). This information is then forwarded to the hairdresser's device.

[1475] hairdresser

[1476] The hairdresser confirms the user's selection and emotional data (input) and begins preparations for the treatment. He / she then consults with the user (output) and provides necessary feedback, such as explaining the feasibility of the selected style and the necessary treatments. After making any final adjustments, the treatment begins.

[1477] Through this series of processes, personalized style suggestions and high-quality treatments are realized based on the user's emotions and personal information.

[1478] (Application example 2)

[1479] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1480] Hair salons need to propose optimal hairstyles based on the individual feelings and preferences of their customers, thereby increasing customer satisfaction. However, conventional systems have the problem that customers cannot confirm which style suits them until they actually visit the salon. Furthermore, there is no way for users to try the style at home before the treatment, and due to a lack of communication with the hairdresser, the desired style may not be fully reflected.

[1481] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1482] In this invention, the server includes: means for inputting a user's facial image and personal information; means for analyzing the facial image and personal information and extracting feature data; means for generating multiple personalized style candidates based on the extracted feature data; means for presenting the generated style candidates to the user in real time; means for notifying the hairdresser of the user's selected style; means for the user to use the virtual beauty salon app at home; means for capturing a facial image and inputting a desired hairstyle using a smartphone; and means for generating a hairstyle in real time using an AI model and an emotion engine through the virtual beauty salon app. This allows the user to try out multiple hairstyles at home and select the best style before visiting the salon. As a result, user satisfaction is improved and communication with the hairdresser is smoother.

[1483] "User" refers to an individual who uses the services of a beauty salon.

[1484] "Facial image" means a photograph or image data of a user's face.

[1485] "Personal Information" refers to your hair type, preferred hairstyles and colors, and other personal information.

[1486] "Feature data" refers to data such as facial shape, hair type, and skin condition analyzed from the input facial image and personal information.

[1487] "Style suggestions" refers to multiple hairstyles suggested to the user.

[1488] "Real-time" means immediate response or processing without delay.

[1489] "Practitioner" refers to a professional who performs hairstyles in a beauty salon, i.e., a hairdresser.

[1490] A "virtual beauty salon app" refers to application software that allows users to try out hairstyles at home using their smartphones.

[1491] "Smartphone" means a portable communication device that has Internet connectivity and can be used by installing applications.

[1492] An "AI model" refers to artificial intelligence that uses technologies such as machine learning and deep learning to analyze a user's characteristic data and generate optimal style candidates.

[1493] "Emotion engine" refers to technology that reads emotions from a user's facial expressions and uses those emotions as data for analysis.

[1494] The system of the present invention provides personalized hairstyle suggestions that combine AI technology and an emotion engine to improve the user experience in beauty salons. This system includes a means for inputting a user's facial image and personal information, a means for analyzing the facial image and personal information and extracting feature data, a means for generating multiple personalized style candidates based on the extracted feature data, a means for presenting the generated style candidates to the user in real time, and a means for notifying the hairdresser of the style results selected by the user.

[1495] 1. Enter and submit user information

[1496] Terminal

[1497] Users use their smartphone to capture a photo of their face at home or at a dedicated terminal in a beauty salon, and input their hair type, preferred hairstyle, and color. The facial image is then captured using the smartphone's camera or the dedicated terminal's camera. This data is encrypted as needed and sent to a server.

[1498] 2. Analysis and style candidate generation

[1499] server

[1500] After receiving the user's personal information and facial image, the server uses an AI model to analyze facial shape, hair type, skin condition, and even emotional data obtained from an emotion engine to extract feature data. Multiple style candidates are then generated based on the extracted feature data. The generated style candidates are then overlaid on the user's facial image to present a realistic image.

[1501] 3. Presentation and selection of style candidates

[1502] Terminal

[1503] The generated style candidates are sent from the server to the device and are presented to the user in real time on a smartphone or dedicated device. The user can then review multiple style candidates applied to their face and select the one that best suits them.

[1504] user

[1505] Users can select their favorite style from the suggested styles, and the system will also incorporate suggestions based on emotional data to optimize the selection.

[1506] 4. Notification of results and feedback

[1507] Terminal

[1508] The results of the selected style are notified to the practitioner in real time, who then confirms the style selected by the user and begins preparations for the treatment.

[1509] Practitioner

[1510] The practitioner will conduct a detailed consultation based on the user's selections, provide feedback as needed, and make any final adjustments before beginning the treatment.

[1511] Hardware and software used

[1512] In this invention, in addition to the camera of a smartphone or dedicated device, a server is used to run the AI ​​model, and existing AI services such as Azure Face API, Google Cloud Vision, and Amazon Rekognition are used for the analysis.

[1513] Specific examples

[1514] For example, when User A uses this system at home, they capture a photo of their face using their smartphone and enter their hair type as "curly" and color as "red." The smartphone encrypts this information and sends it to the server. The server-side AI model analyzes this data, generates multiple personalized style options for User A, and sends the results to the smartphone. Once User A selects the optimal style, the results are notified to the hairdresser in real time.

[1515] Prompt Sentence Examples

[1516] "Generate the best hairstyle candidates for the user based on their face image and preferred hairstyle information. The hair type is "curly hair", the hair color is "red", and the specific face image is attached below."

[1517] The above is a detailed description of the embodiment of the present invention. This system allows users to try out multiple hairstyles at home and choose the best one before visiting a beauty salon. As a result, user satisfaction is improved and communication with the hairdresser is smoother.

[1518] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1519] Step 1:

[1520] Users use a smartphone or dedicated device to capture an image of their face and enter personal information.

[1521] Specific operation: The user activates the smartphone camera and takes a picture of their face, and also inputs their hair type (e.g., curly hair), preferred hairstyle, and color (e.g., red).

[1522] Input: User's face image, hair type, preferred hairstyle and color

[1523] Output: Captured face image and input personal information

[1524] Step 2:

[1525] The facial image and personal information acquired by the terminal are transmitted to the server.

[1526] Specific operation: Using the device's built-in transmission function, the encrypted facial image and personal information are uploaded to the server.

[1527] Input: Captured face image and input personal information

[1528] Output: Facial image and personal information sent to the server

[1529] Step 3:

[1530] The server analyzes the received facial image and personal information.

[1531] How it works: Based on the data received by the server, it uses an AI model and emotion engine to extract facial shape, hair type, skin condition, and emotion data.

[1532] Input: Submitted face image and personal information

[1533] Output: Analyzed feature data (face shape, hair type, skin condition, emotion data)

[1534] Step 4:

[1535] The server generates multiple personalized style candidates based on the extracted feature data.

[1536] How it works: Using an AI model, it generates multiple hairstyles that best fit the user's feature data.

[1537] Input: Parsed feature data

[1538] Output: Image data of style candidates

[1539] Step 5:

[1540] The generated style candidates are sent to the device and presented to the user in real time.

[1541] Specific operation: The server sends image data of the generated style candidates to the terminal, which then displays them on the screen.

[1542] Input: Image data of generated style candidates

[1543] Output: Style suggestions presented to the user

[1544] Step 6:

[1545] The user selects their favorite style from the suggested styles.

[1546] What it does: The user selects the best style from the displayed options and confirms the selection.

[1547] Input: suggested style suggestions

[1548] Output: User selected style

[1549] Step 7:

[1550] The terminal notifies the practitioner of the style results selected by the user.

[1551] Specific operation: The selected style results are sent to the practitioner using the device's built-in notification function.

[1552] Input: User selected style

[1553] Output: Style results communicated to the practitioner

[1554] Step 8:

[1555] The therapist checks the selected style results and the user's emotional data, and begins preparations for the treatment.

[1556] Specific actions: The practitioner prepares for the actual treatment based on the notified style results, and also conducts detailed consultations with the user and provides feedback as needed.

[1557] Input: Notified style results and emotion data

[1558] Output: Prepared treatment and provided feedback

[1559] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the control target 443 to output the result of the specific processing. The microphone 238 acquires voice indicating a user input regarding the result of the specific processing. The control unit 46A transmits voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[1560] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1561] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the robot 414.

[1562] The emotion identification model 59 as an emotion engine may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to an emotion map (see FIG. 9), which is a specific mapping. Similarly, the emotion identification model 59 may determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[1563] FIG. 9 is a diagram illustrating an emotion map 400 on which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. Emotions closer to the center of the concentric circles are more primitive. Emotions representing states and actions arising from a state of mind are arranged on the outer edges of the concentric circles. The concept of emotion includes both affect and mental states. Emotions generally generated from reactions occurring in the brain are arranged on the left side of the concentric circles. Emotions generally induced by situational judgment are arranged on the right side of the concentric circles. Emotions generally generated from reactions occurring in the brain and induced by situational judgment are arranged on the upper and lower sides of the concentric circles. Furthermore, the emotion of "pleasure" is arranged on the upper side of the concentric circles, and the emotion of "discomfort" is arranged on the lower side. In this way, in the emotion map 400, multiple emotions are mapped based on the structure by which emotions are generated, and emotions that tend to occur simultaneously are mapped close to each other.

[1564] These emotions are distributed in the 3 o'clock direction on emotion map 400, and typically fluctuate between relief and anxiety. In the right half of emotion map 400, situational awareness dominates over internal sensations, resulting in a sense of calm.

[1565] The inside of emotion map 400 represents what is going on in the mind, and the outside of emotion map 400 represents behavior, so the further you go outside emotion map 400, the more visible the emotions become (the more they are expressed in behavior).

[1566] Human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. Emotions can also be created for robots, automobiles, and motorcycles, based on various balances, such as posture and remaining battery life. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. An emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on Voice Emotion Recognition and Emotional Brain Physiological Signal Analysis Systems, Tokushima University, Doctoral Dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map lists emotions belonging to the "reaction" domain, where sensation is dominant. The right half of the emotion map lists emotions belonging to the "situation" domain, where situational awareness is dominant.

[1567] The emotion map defines two emotions that promote learning. One is a negative emotion on the situation side, around the middle of "repentance" or "reflection." In other words, this occurs when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is a positive emotion on the response side, around "desire." In other words, this occurs when the robot experiences positive feelings such as "I want more" or "I want to know more."

[1568] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​indicating each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple pieces of training data that are combinations of user input and emotion values ​​indicating each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions that are located close to each other have similar values, as in the emotion map 900 shown in FIG. 10. FIG. 10 shows an example in which multiple emotions, "relieved," "calm," and "reassuring," have similar emotion values.

[1569] The system according to the present disclosure has been described above mainly with respect to the functions of the data processing device 12, but the system according to the present disclosure is not necessarily implemented on a server. The system according to the present disclosure may be implemented as a general information processing system. The present disclosure may be implemented, for example, as a software program running on a personal computer or an application running on a smartphone, etc. The method according to the present disclosure may be provided to users in the form of SaaS (Software as a Service).

[1570] In the above embodiment, an example was given in which the specific processing is performed by one computer 22, but the technology of the present disclosure is not limited to this, and the specific processing may be distributed and performed by a plurality of computers including the computer 22. For example, the data generation model 58 may be provided in an external device of the data processing device 12, and data may be generated in the external device in accordance with input data.

[1571] In the above embodiment, an example in which the specific processing program 56 is stored in the storage 32 has been described, but the technology of the present disclosure is not limited to this. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-transitory storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-transitory storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes the specific processing in accordance with the specific processing program 56.

[1572] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[1573] It is not necessary to store all of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store all of the specific processing program 56 in the storage 32; only a portion of the specific processing program 56 may be stored.

[1574] The hardware resource for executing a specific process can be any of the following processors: An example of a processor is a CPU, which is a general-purpose processor that functions as a hardware resource for executing a specific process by executing software, i.e., a program. Another example of a processor is a dedicated electrical circuit, such as an FPGA (Field-Programmable Gate Array), a PLD (Programmable Logic Device), or an ASIC (Application Specific Integrated Circuit), which is a processor with a circuit configuration designed specifically for executing a specific process. Each processor has built-in or connected memory, and each processor uses the memory to execute the specific process.

[1575] The hardware resource that executes the specific processing may be configured with one of these various processors, or may be configured with a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Also, the hardware resource that executes the specific processing may be a single processor.

[1576] As an example of a system configured with a single processor, first, one processor is configured by combining one or more CPUs and software, and this processor functions as a hardware resource that executes a specific process. Second, there is a system that uses a processor that realizes the functions of an entire system including multiple hardware resources that execute a specific process on a single IC chip, as typified by SoC (System-on-a-chip). In this way, a specific process is realized using one or more of the above-mentioned various processors as hardware resources.

[1577] Furthermore, the hardware structure of these various processors can be, more specifically, an electric circuit that combines circuit elements such as semiconductor devices. The specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps may be deleted, new steps may be added, or the processing order may be rearranged, without departing from the spirit of the invention.

[1578] The above-described description and illustrations are a detailed explanation of the parts related to the technology of the present disclosure and are merely an example of the technology of the present disclosure. For example, the above description of the configuration, functions, actions, and effects is an explanation of an example of the configuration, functions, actions, and effects of the parts related to the technology of the present disclosure. Therefore, it goes without saying that unnecessary parts may be deleted, new elements may be added, or replacements may be made to the above-described description and illustrations within the scope of the gist of the technology of the present disclosure. Furthermore, to avoid confusion and facilitate understanding of the parts related to the technology of the present disclosure, the above-described description and illustrations omit explanations of common technical knowledge that do not require particular explanation to enable the implementation of the technology of the present disclosure.

[1579] All publications, patent applications, and technical standards mentioned in this specification are herein incorporated by reference to the same extent as if each individual publication, patent application, or technical standard was specifically and individually indicated to be incorporated by reference.

[1580] The following is further disclosed regarding the above embodiment.

[1581] (Claim 1)

[1582] a means for inputting a user's facial image and personal information;

[1583] A means for analyzing a face image and personal information and extracting feature data;

[1584] a means for generating a plurality of personalized style candidates based on the extracted feature data;

[1585] A means for presenting the generated style candidates to the user in real time;

[1586] means for notifying the hairdresser of the style results selected by the user;

[1587] A system including:

[1588] (Claim 2)

[1589] 10. The system of claim 1, further comprising means for providing style and color suggestions based on the user's skin condition.

[1590] (Claim 3)

[1591] 10. The system of claim 1, further comprising means for a hairdresser to provide feedback to a user on a result of selection from the presented style candidates.

[1592] "Example 1"

[1593] (Claim 1)

[1594] a means for inputting a user's facial image and personal information;

[1595] A means for encrypting the face image and personal information and transmitting them to a server;

[1596] means for decrypting the encrypted data in the server and extracting the feature data;

[1597] A means for generating a plurality of personalized style candidates using a generative AI model based on the extracted feature data;

[1598] A means for presenting the generated style candidates to the user in real time;

[1599] a means for notifying the provider of the style results selected by the user;

[1600] A system including:

[1601] (Claim 2)

[1602] 10. The system of claim 1, further comprising means for providing style and color suggestions based on the user's skin condition.

[1603] (Claim 3)

[1604] 10. The system according to claim 1, further comprising means for a provider to provide feedback on a result of a user's selection from the presented style candidates.

[1605] "Application Example 1"

[1606] (Claim 1)

[1607] a device for inputting a user's face image and personal information;

[1608] A device that analyzes a face image and personal information and extracts feature data;

[1609] a device for generating a plurality of personalized style candidates based on the extracted feature data;

[1610] a device that presents the generated style candidates to a user in real time;

[1611] a device for notifying a technician of the style results selected by the user;

[1612] a device for transmitting the selected style, presenting the style in real time, and generating prompts for accepting the selection;

[1613] A system including:

[1614] (Claim 2)

[1615] 10. The system of claim 1, further comprising a device that provides style and color suggestions based on the user's skin condition.

[1616] (Claim 3)

[1617] 10. The system of claim 1, further comprising a device for a technician to provide feedback to a user on a selection made by the user from the presented style candidates.

[1618] "Example 2: Combining Emotion Engines"

[1619] (Claim 1)

[1620] a means for inputting a user's facial image and personal information;

[1621] A means for analyzing a face image and personal information and extracting feature data;

[1622] a means for generating a plurality of personalized style candidates based on the extracted feature data;

[1623] A means for presenting the generated style candidates to the user in real time;

[1624] means for notifying the hairdresser of the style results selected by the user;

[1625] A means for acquiring user emotion data using an emotion engine and reflecting the data in generating style candidates;

[1626] A system including:

[1627] (Claim 2)

[1628] 10. The system of claim 1, wherein the system provides style and color suggestions based on the user's skin condition.

[1629] (Claim 3)

[1630] 10. The system of claim 1, wherein a hairdresser provides feedback to a user regarding a selection made from presented style candidates.

[1631] "Application example 2 when combining emotion engines"

[1632] (Claim 1)

[1633] a means for inputting a user's facial image and personal information;

[1634] A means for analyzing a face image and personal information and extracting feature data;

[1635] a means for generating a plurality of personalized style candidates based on the extracted feature data;

[1636] A means for presenting the generated style candidates to the user in real time;

[1637] a means for notifying the practitioner of the style results selected by the user;

[1638] A means for users to use the virtual beauty salon app at home;

[1639] A means for capturing a face image and inputting a preferred hairstyle using a smartphone;

[1640] A means to generate hairstyles in real time using AI models and emotion engines through a virtual hair salon app;

[1641] A system including:

[1642] (Claim 2)

[1643] 10. The system of claim 1, further comprising means for providing style and color suggestions based on the user's skin condition.

[1644] (Claim 3)

[1645] 10. The system of claim 1, further comprising means for a practitioner to provide feedback to a user's selection from the presented style candidates. [Explanation of symbols]

[1646] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Device 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robot< / url:> < / url:> < / url:> < / url:>

Claims

1. a means for inputting a user's facial image and personal information; A means for analyzing a face image and personal information and extracting feature data; a means for generating a plurality of personalized style candidates based on the extracted feature data; A means for presenting the generated style candidates to the user in real time; means for notifying the hairdresser of the style results selected by the user; A system including:

2. 10. The system of claim 1, further comprising means for providing style and color suggestions based on the user's skin condition.

3. 10. The system of claim 1, further comprising means for a hairdresser to provide feedback to a user on a result of selection from the presented style candidates.

Images