system

Abstract

We provide the system. [Solution] A means for analyzing facial images received from users and information on desired hairstyles, A means for presenting multiple hairstyle candidates generated based on the analysis results, A means of providing selected hairstyle information to beauty professionals, A system that includes this.

Description

Technical Field

[0001] The technology of the present disclosure relates to a system.

Background Art

[0002] Patent Document 1 discloses a method for controlling a persona chatbot, which is performed by at least one processor, and includes steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to an explanation of the chatbot's character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] When requesting a hairstyle at a beauty salon or the like, there may be a gap between the finish desired by the user and the actual result, and this gap may cause dissatisfaction among users. Also, it is difficult to provide an appropriate style when individual conditions such as the hair characteristics and allergies of each user are not considered. An object of the present invention is to solve such problems and provide an optimal hairstyle based on the user's wishes and hair characteristics.

Means for Solving the Problems

[0005] This invention provides means for analyzing a facial image received from a user and desired hairstyle information, for presenting multiple hairstyle candidates generated based on the analysis results, and for providing the selected hairstyle information to a beauty professional, thereby generating an optimal style that suits the individual hair characteristics of each user. Furthermore, it improves the safety of the hair styling service by registering the user's allergy information and selecting appropriate chemicals based on the registered information.

[0006] A "user" refers to an individual who uses the system to select a hairstyle and provides that information.

[0007] A "face image" refers to a digital image that captures the features of the user's face, and is used by the system to analyze the user's hair characteristics.

[0008] "Hairstyle information" refers to data about the design and style of the hairstyle desired by the user, and serves as the basis for the system to generate the optimal style.

[0009] "Means of analysis" refers to software or hardware functions that determine the individual hair characteristics of a user based on facial images and hairstyle information.

[0010] "Generated hairstyle options" refers to multiple hairstyle options suggested by the system based on the user's hair characteristics and preferences.

[0011] A "beauty professional" refers to a professional who performs haircuts and styling in a beauty salon, and who performs treatments based on style information provided by a system.

[0012] "Allergy information" refers to data on a user's sensitivity to specific chemicals or medications, and is used as a safety measure when providing hairstyles. [Brief explanation of the drawing]

[0013] [Figure 1] This is a conceptual diagram showing an example of the configuration of a data processing system according to the first embodiment. [Figure 2] This is a conceptual diagram showing an example of the essential functions of a data processing device and a smart device according to the first embodiment. [Figure 3] This is a conceptual diagram showing an example of the configuration of a data processing system according to the second embodiment. [Figure 4] This is a conceptual diagram showing an example of the main functions of a data processing device and smart glasses according to the second embodiment. [Figure 5] This is a conceptual diagram showing an example of the configuration of a data processing system according to the third embodiment. [Figure 6] This is a conceptual diagram showing an example of the main functions of a data processing device and a headset-type terminal according to the third embodiment. [Figure 7] This is a conceptual diagram showing an example of the configuration of a data processing system according to the fourth embodiment. [Figure 8] This is a conceptual diagram showing an example of the main functions of a data processing device and a robot according to the fourth embodiment. [Figure 9] This shows an emotion map where multiple emotions are mapped. [Figure 10] This shows an emotion map where multiple emotions are mapped. [Figure 11] This is a sequence diagram showing the processing flow of the data processing system in Example 1. [Figure 12] This is a sequence diagram showing the processing flow of the data processing system in Application Example 1. [Figure 13] This is a sequence diagram showing the processing flow of the data processing system in Example 2, which incorporates an emotion engine. [Figure 14] This is a sequence diagram showing the processing flow of the data processing system in Application Example 2, which combines an emotion engine. [Modes for carrying out the invention]

[0014] An example of an embodiment of the system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0015] First, the terms used in the following description will be explained.

[0016] In the following embodiments, the numbered processor (hereinafter simply referred to as "processor") may be a single arithmetic unit or a combination of multiple arithmetic units. Also, the processor may be a single type of arithmetic unit or a combination of multiple types of arithmetic units. Examples of arithmetic units include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), an APU (Accelerated Processing Unit), and the like.

[0017] In the following embodiments, the numbered RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a work memory by the processor.

[0018] In the following embodiments, the numbered storage is one or more non-volatile storage devices that store various programs and various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), or magnetic tapes, etc.

[0019] In the following embodiments, the signed communication interface (I / F) is an interface that includes a communication processor and an antenna, etc. The communication interface manages communication between multiple computers. Examples of communication standards applicable to the communication interface include wireless communication standards such as 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), or Bluetooth (registered trademark).

[0020] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." That is, "A and / or B" means that it may be A alone, or B alone, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" applies when expressing three or more things linked by "and / or."

[0021] [First Embodiment]

[0022] Figure 1 shows an example of the configuration of the data processing system 10 according to the first embodiment.

[0023] As shown in Figure 1, the data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0024] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0025] The smart device 14 comprises a computer 36, a reception device 38, an output device 40, a camera 42, and a communication interface 44. The computer 36 comprises a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The reception device 38, output device 40, and camera 42 are also connected to the bus 52.

[0026] The reception device 38 is equipped with a touch panel 38A and a microphone 38B, etc., and receives user input. The touch panel 38A receives user input by detecting contact with an object (e.g., a pen or finger). The microphone 38B receives user input by detecting the user's voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0027] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form perceptible to the user 20 (e.g., audio and / or text). The display 40A displays visible information such as text and images according to instructions from the processor 46. The speaker 40B outputs audio according to instructions from the processor 46. The camera 42 is a small digital camera equipped with an optical system such as a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0028] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various types of information between processor 46 and processor 28 via network 54.

[0029] Figure 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0030] As shown in Figure 2, in the data processing device 12, a specific processing is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" related to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 according to the specific processing program 56 executed on the RAM 30.

[0031] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0032] In the smart device 14, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The reception output program 60 is used in conjunction with a specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0033] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0034] This invention relates to a method and apparatus for implementing an "AI haircut ordering system." The system generates an optimal hairstyle based on the user's facial image and desired hairstyle information, and presents it to the user in advance, thereby reducing the gap between the user's wishes and the actual finished look.

[0035] User actions

[0036] First, the user downloads and launches a dedicated application on their device. Using the application, the user takes a photo of their face and selects their desired hairstyle. This information is then used as input for the system.

[0037] Terminal processing

[0038] The device generates a data packet to send a facial image and desired hairstyle information to the server. The data packet includes the user's individual facial image and information about the style selected by the user. If the user's allergy information has been registered in advance, that information is also included.

[0039] Server Processing

[0040] The server analyzes the received data and uses an algorithm to analyze facial image features to extract the user's facial contours and hair characteristics. Next, a generative AI model is used to generate multiple hairstyle candidates based on the analyzed features. This generative AI model has been trained on past hairstyle datasets and the latest trend information, and provides the optimal candidate based on this.

[0041] The multiple style options generated from the analysis results are sent back to the terminal for user confirmation. The user can select the style they like best from the presented options. This selected style is recorded as the final order and provided to the beauty professional.

[0042] Specific example

[0043] For example, if a user wants a "short bob style," they log into the system, upload a photo of their face, search for "short bob," and select their favorite style. The system analyzes the facial image, generates several suitable bob style variations, and presents them to the user. The style B selected by the user is ultimately recorded and provided to a beauty professional along with related information.

[0044] This system allows users to see an image of the finished look beforehand, minimizing the gap between their expectations and the final result, and making their salon experience more satisfying.

[0045] The following describes the processing flow.

[0046] Step 1:

[0047] The user launches a smartphone application, follows the instructions to take a picture of their face, and searches for and selects their preferred hairstyle. The image and selected style information are recorded on the device.

[0048] Step 2:

[0049] The device generates facial images and hairstyle information as data packets and sends them to the server. The data may also include the user's allergy information.

[0050] Step 3:

[0051] The server analyzes the received data, examining the user's facial contours and hair texture from the facial image. A facial recognition algorithm is used to extract detailed hair characteristics.

[0052] Step 4:

[0053] The server runs an AI model and generates multiple hairstyle options that suit the user's characteristics based on the analysis results. This model takes trend information into account and suggests the optimal style.

[0054] Step 5:

[0055] The server sends the generated hairstyle options to the terminal.

[0056] Step 6:

[0057] The device presents the user with several style options it has received. The user reviews them and selects the style they like best.

[0058] Step 7:

[0059] The terminal records the style information selected by the user as the final order and sends it to a server for communication with beauty professionals.

[0060] Step 8:

[0061] The server provides beauty professionals with details on the selected style, the treatment method, and the necessary products. This ensures that the treatment at the salon runs smoothly.

[0062] (Example 1)

[0063] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0064] It is essential to reduce the gap between the hairstyle the user desires and the actual finished look, and to provide the optimal style that suits each individual's characteristics. Furthermore, it is necessary to perform safe and effective treatments based on the user's health information.

[0065] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0066] In this invention, the server includes means for analyzing facial images received from the user, desired hairstyle information, and registered health information; means for presenting multiple hairstyle candidates generated using the analysis results and a generation AI model; and means for providing the hairstyle information selected by the user to a specialist to support the provision of optimal service. This makes it possible to suggest a style suitable for the user and to provide safe and highly satisfying beauty services.

[0067] A "user" refers to an individual who uses the system to receive suggestions and make selections for their own hairstyle.

[0068] A "face image" is digital image data that includes the features of the user's face.

[0069] "Hair style information" refers to digital data and attribute information related to the hairstyle desired by the user.

[0070] "Health information" refers to information related to the user's health status and allergies.

[0071] "Analysis method" refers to a method of processing facial images and hairstyle information based on received data and extracting their characteristics.

[0072] A "generative AI model" is an artificial intelligence program that creates optimal hairstyle options for a user based on past data and trends.

[0073] "Presentation method" refers to the method of displaying the analyzed and generated hairstyle candidates to the user.

[0074] A "specialist" refers to a professional who possesses the skills and knowledge related to beauty and provides treatments and advice to users.

[0075] This section describes the embodiments for carrying out the invention. This system, an "AI haircut ordering system," utilizes smart devices and computer servers to facilitate end-user use. The overall system overview and operation are described in detail below.

[0076] Users access the platform by downloading a dedicated application to their smart device and launching it. Within the application, users take a photo of their face using the camera and select their desired hairstyle. This automatically sends the user's face image and hairstyle information to the system.

[0077] The device generates a data packet containing the captured facial image and the user's selected hairstyle information, and prepares it for transmission to the server. This data packet may also include pre-registered health information.

[0078] The server analyzes the received data packets. On the server, image analysis software (e.g., OpenCV or DeepFace) is used to extract features from facial images. Based on the analyzed data, a generative AI model (e.g., an AI program built with TENSORFLOW®) is used to generate various hairstyle options. This AI model has been trained on historical data and the latest trends, and can present the optimal style based on the user's preferences and characteristics.

[0079] The generated hairstyle options are sent to the user's device, allowing the user to select the style that best suits their preferences from among several options. This selected information is shared with beauty professionals and related staff and used to provide the user with the most suitable beauty services.

[0080] As a concrete example, after a user logs into the app, they upload a photo of themselves using the "camera" function, and then search for and select a "short bob" style. Based on this information, the server inputs a prompt message to the AI ​​model saying, "Generate a short bob style that suits these facial features," and generates an appropriate style.

[0081] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0082] Step 1:

[0083] The user launches a dedicated application on their smart device, takes a picture of their face with the camera, and selects their desired hairstyle. The input includes the facial image data the user has captured and the selected hairstyle information. This information is saved on the device based on the user's actions.

[0084] Step 2:

[0085] The device generates a data packet containing saved facial images and hairstyle information, as well as health information previously registered by the user. This data packet is structured in an appropriate format (e.g., JSON format) and prepared for transmission to the server.

[0086] Step 3:

[0087] The server analyzes data packets received from the terminal. The input is data packets, and the facial images contained within them are processed using image analysis software (e.g., OpenCV) to extract the user's facial features. The output is a feature vector containing facial contours and hair texture information.

[0088] Step 4:

[0089] The server uses a generative AI model to generate hairstyle candidates based on extracted facial features and input hairstyle information. The input consists of feature vectors, hairstyle information, and a prompt message, such as "Generate hairstyles that match these facial features." The output includes image data of multiple hairstyle candidates.

[0090] Step 5:

[0091] The image data of hairstyle candidates generated on the server is sent back to the terminal for user confirmation. The terminal displays the candidate styles on the screen for the user. Based on this information, the user can select the most suitable hairstyle.

[0092] Step 6:

[0093] The hairstyle information selected by the user is recorded as the final order and provided to the stylist. The server transmits this information to the stylist in the appropriate format to help prepare the beauty service for the user.

[0094] (Application Example 1)

[0095] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0096] Traditionally, hairstyle simulations and selections have often relied on static images, leading to discrepancies between the simulated look and the actual finished style. This makes it difficult for users to achieve satisfactory results in salons, resulting in dissatisfaction. Furthermore, there is a lack of sufficient means for users to visually try out suitable styles in real time.

[0097] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0098] In this invention, the server includes means for analyzing facial information received from the user and the selected appearance information; means for presenting a plurality of appearance candidates generated based on the analysis results; means for providing the selected appearance information; and means for displaying the change in appearance using the user's head image. This allows the user to visually try out different styles in real time using a smart device and select the appearance that best matches their preferences.

[0099] "Facial information received from the user" refers to image data of the user's face, which is used to analyze facial features.

[0100] "Selected appearance information" refers to information about the appearance and style desired by the user, and is data used to select appearances.

[0101] "Means of analysis" refers to the processes and technologies used to determine the suitability of a user's characteristics and style based on the received data.

[0102] "Means for presenting multiple candidate appearances" refers to a method that has the function of visually showing the user multiple styles generated based on the analysis results.

[0103] "Means of providing selected appearance information" refers to a system that stores data on the appearance selected by the user and provides it to external assistance as needed.

[0104] A "device that displays changes in appearance using a user's head image" is a device that projects changes in appearance onto a real image of the user's head, providing a visual simulation.

[0105] This invention provides an interactive appearance simulation system. The server performs an appearance simulation using a face image received from the user's smart device and selected appearance information. The server analyzes the face image using image processing software such as OpenCV and TensorFlow for executing generative AI models, and extracts individual user features. This makes it possible to identify characteristics such as the shape and dimensions of the user's face and hair texture.

[0106] Based on the identified features, the server uses a pre-trained AI model to generate multiple appearance candidates that match those features. These candidates reflect the latest trends, providing users with a variety of choices.

[0107] The device displays generated appearance options in real time on the smart glasses' display. Users can visually try out various appearances through the glasses and easily switch between them using voice and eye-tracking sensors.

[0108] For example, if a user wants to try a "casual look," they can use a prompt to instruct the AI ​​model to "generate multiple variations of casual styles from the user's facial image and display them in real time." Following this prompt, the AI ​​will present the user with the most suitable look options, which the user can then review and select their preferred look. This system allows users to easily try out different looks in a physical store and choose the one they like best, resulting in a highly satisfying experience.

[0109] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0110] Step 1:

[0111] The user takes a facial image using a smart device and selects desired appearance information. This information is entered into the device. The device then generates a data packet containing the facial image and selected appearance information and sends it to the server.

[0112] Step 2:

[0113] The server extracts the user's face image and appearance information from the received data packets. The server uses OpenCV to analyze the face image and identify facial features. Specifically, it detects the shape, dimensions, and hair texture of the face. The output of this process is a numerical representation of the facial features.

[0114] Step 3:

[0115] The server inputs the obtained facial features into a generating AI model and generates appearance candidates based on the prompt "Generate multiple appearance variations from the user's face image." The AI ​​model considers pre-trained trend information and outputs the most suitable appearance.

[0116] Step 4:

[0117] The generated appearance options are sent to the terminal by the server. The terminal displays the appearance changes in real time on the smart glasses' display. The user can visually confirm the appearance through the glasses and switch between options using gaze or voice commands.

[0118] Step 5:

[0119] The user's final selected appearance information is recorded on the device and uploaded to the server. This recorded information is then provided to the relevant service providers and used for future service improvements and personalized support.

[0120] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0121] This invention relates to an "AI haircut ordering system" that combines a "emotion engine" that acquires a user's facial image and desired hairstyle information and recognizes the user's emotions. The system generates hairstyle candidates that take into account the user's hair characteristics and emotions, and provides them to beauty professionals, thereby providing a service that fits the user's wishes and emotions.

[0122] User actions

[0123] The user downloads and launches a smartphone application. After launching, they use their device to take a photo of their face and select their desired hairstyle. The application uses this information as system input.

[0124] Terminal processing

[0125] The device forms a data packet containing the captured facial image and the user's selected hairstyle information, and sends it to the server. If the user's allergy information has been registered in advance, that information will also be included in the data.

[0126] Processing by the emotion engine

[0127] The server uses an emotion engine to analyze received facial images, determining the user's emotional state from their facial expressions. This emotion recognition helps evaluate whether the style chosen by the user fits their emotional state.

[0128] Server Processing

[0129] Taking into account the user's emotions recognized by the emotion engine, the server uses a generative AI model to generate multiple hairstyle options that reflect the user's characteristics. The order in which the generated styles are presented may be adjusted based on the emotions. Information suggesting precautions for beauty professionals during treatment can also be generated.

[0130] Presenting the style

[0131] The terminal presents the user with multiple style options received from the server. The user selects their favorite style from the presented options, which is then recorded as the final order and provided to the beauty professional.

[0132] Specific example

[0133] For example, if a user requests "long curls," but the emotion engine recognizes from the user's current facial expression that they want to "relax," the system will prioritize suggesting styles that emphasize "relaxation." The style chosen by the user is recorded as the final order, and during the treatment, the beauty professional is provided with notes based on the emotion recognition.

[0134] This system allows us to provide hairstyle services that suit both the user's preferences and emotional state, thereby improving the user experience.

[0135] The following describes the processing flow.

[0136] Step 1:

[0137] The user launches a smartphone application, follows the instructions to take a picture of their face, and selects their desired hairstyle. The face image and selected style information are then entered into the device.

[0138] Step 2:

[0139] The device forms a data packet containing the captured facial image and selected hairstyle information, and sends it to the server. The data may also include the user's allergy information.

[0140] Step 3:

[0141] The server analyzes the received facial image using an emotion engine, recognizing the user's emotional state from their facial expressions. This allows for responses tailored to the user's emotions, such as a style that makes them feel safe and relaxed, or a style that energizes them.

[0142] Step 4:

[0143] The server analyzes hair characteristics based on the facial image and selected hairstyle information, and uses a generative AI model to generate multiple hairstyle candidates suitable for the user. The priority of the style candidates is adjusted based on the recognized emotional state.

[0144] Step 5:

[0145] The server sends the generated hairstyle suggestions and their detailed information to the terminal. This includes style images, key cutting points, recommended products, and emotionally-driven suggestions.

[0146] Step 6:

[0147] The device presents the user with multiple style options it has received. The user selects the style that seems to fit best from the presented options. This selection is recorded as the final order.

[0148] Step 7:

[0149] The selected style information is sent again from the terminal to the server and provided to the beauty professional as the final decision.

[0150] Step 8:

[0151] The server provides beauty professionals with detailed information, including the selected style and considerations based on the user's emotional state to be taken into account when performing the treatment. This ensures that the treatment aligns with the user's emotions, leading to increased satisfaction.

[0152] (Example 2)

[0153] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0154] Traditionally, there have been no systems that consider the user's emotional state when selecting a hairstyle, resulting in users not achieving their desired level of satisfaction. Furthermore, in order to suggest an appropriate style, it is necessary to accurately reflect information about each individual's head characteristics and health, but previous systems have not adequately addressed this issue.

[0155] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0156] In this invention, the server includes means for analyzing a facial image received from the user and selected style information, means for recognizing the user's emotional state from the facial image using an emotion engine, and means for a generating AI model to generate and present multiple style candidates based on the analysis results and emotional state. This makes it possible to suggest styles suitable for individual characteristics while taking into account the user's emotional state, and further select appropriate products and services based on health information.

[0157] A "user" refers to an individual who uses the system to provide their facial image and style information in order to receive beauty services.

[0158] "Facial images" refer to image data captured in order to analyze a user's facial expressions and head characteristics.

[0159] "Style information" refers to information about the hairstyle desired by the user, including the type and characteristics of the selected style.

[0160] An "emotion engine" refers to software or algorithms that analyze a user's facial image and recognize their emotional state from their facial expressions.

[0161] A "generative AI model" refers to artificial intelligence technology that generates optimal style candidates based on the user's characteristics and emotional state.

[0162] "Hairstyle options" refers to the multiple hairstyle choices suggested to the user, from which the user selects the one that best suits them.

[0163] "Health-related information" refers to allergy information and other health status data provided by users, which is used to select appropriate products and services.

[0164] A "beauty professional" refers to a specialized service provider who performs hair styling treatments for users.

[0165] The specific operation of the system in an embodiment for carrying out this invention is shown below.

[0166] First, the user launches a dedicated application installed on their mobile device. Using the application, the user can take a photo of their face and input their desired style information. This face photo and style information are stored on the device as data packets. If necessary, the user can register health information in the application and record allergies and special precautions.

[0167] The terminal sends the generated data packets to the server. This transmission takes place via the internet, ensuring high speed and security. The server is equipped with advanced image analysis software and an emotion engine, which evaluates the user's emotional state from the received facial images. This emotion engine is based on algorithms that analyze the user's facial features and identify emotions in real time.

[0168] Furthermore, the server utilizes a generative AI model to generate multiple style options that suit the user's characteristics and emotions. These generated style options are then presented in a prioritized manner based on the user's emotional state. For example, if the server determines that the user desires "long curls" but also wants to relax, it will prioritize suggesting styles that evoke a strong sense of relaxation.

[0169] Style information transmitted from the server is displayed on the terminal. The user can view style options on the screen and select their preferred one. The selected style is recorded as the final choice and, when provided to a beauty professional, is accompanied by treatment advice based on the emotional criteria of the suggested style.

[0170] Throughout this entire process, users can receive personalized beauty services tailored to their emotional state and preferences.

[0171] An example of a prompt message might be, "Suggest a hairstyle suitable for a user who wants to relax." This allows the system to generate and provide a style that incorporates the user's emotions and desires.

[0172] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0173] Step 1:

[0174] The user launches an application installed on their mobile device and takes a photo of their face. The application provides an input screen where the user can select their desired hairstyle. This input process yields both the facial image data and the selected style information.

[0175] Step 2:

[0176] The terminal composes the input facial image data and style information into a data packet. This data packet also includes health information registered by the user, if necessary. The terminal prepares to transmit this data packet to the server via the network connection.

[0177] Step 3:

[0178] The server receives data packets over the network. Based on the received data, it processes the facial image using image analysis software to extract the user's facial features. This outputs facial feature quantities from the image data for input into the emotion engine.

[0179] Step 4:

[0180] The server inputs the extracted facial features into the emotion engine. The emotion engine uses these features to recognize the user's emotional state. This process allows for the recognition of emotional states such as "joy" and "relaxation."

[0181] Step 5:

[0182] The server inputs emotional state and style information into a generative AI model, which generates style candidates suitable for the user's characteristics and emotional state. The generative AI model uses machine learning techniques to output multiple style candidates from this data.

[0183] Step 6:

[0184] The server prioritizes the generated style candidates based on the user's emotional state. For example, if the user is perceived as wanting to relax, styles that emphasize "relaxation" will be prioritized. This prioritized style information is then output.

[0185] Step 7:

[0186] The terminal presents the user with style options received from the server. The user can view multiple style options on the screen and select the one they like best. This selection then provides the final selected style information.

[0187] Step 8:

[0188] The terminal sends the user's selected style information to the server as the final order. The server provides the beauty professional with treatment considerations based on the selected style and emotions, helping them to provide a service that matches the user's wishes and emotions.

[0189] (Application Example 2)

[0190] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0191] Conventional hairstyle suggestion systems have limitations in improving the user experience because they make suggestions without considering the user's emotional state. Furthermore, it was difficult to suggest the most suitable hairstyle to the customer in real time. These challenges made it difficult to provide services and customer care that would satisfy users.

[0192] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0193] In this invention, the server includes means for analyzing a facial image received from the user and information on the desired hairstyle, means for analyzing the user's emotional state in real time using a visual device, and means for adjusting suggested hairstyles based on the user's emotional state. This makes it possible to suggest hairstyles that are suitable for the user's emotions.

[0194] "Means for analyzing facial images and desired hairstyle information received from a user" refers to a processing device that uses digital technology to analyze facial image data and information about desired hairstyles obtained from a user, and extracts style candidates based on the results.

[0195] "Means for presenting multiple hairstyle options generated based on analysis results" refers to an interface for visually or electronically suggesting multiple hairstyles generated from the analysis results to the user.

[0196] "Means for providing selected hairstyle information to beauty professionals" refers to an information provision device that transmits information about the hairstyle selected by the user to beauty professionals, enabling them to perform treatments based on that information.

[0197] "Means for analyzing a user's emotional state in real time using visual devices" refers to a technology that uses visual devices such as cameras to instantly determine emotions from a user's facial expressions and other visual information, and then analyzes that information.

[0198] "Means for adjusting suggested hairstyles based on the user's emotional state" refers to an algorithm or system that adjusts the priority and content of suggested hairstyles according to the results of emotion analysis, thereby providing the user with the most suitable option based on their mood and emotions.

[0199] In implementing this invention, the user first accesses the system using a terminal device such as a smartphone or tablet. The user takes a picture of their face and selects their desired hairstyle. This information is digitized on the terminal and transmitted to the server. The server utilizes image recognition technology and sentiment analysis software to analyze the received face image and hairstyle information.

[0200] Specifically, the server analyzes the user's face using a facial recognition API (e.g., AWS® Rekognition) and evaluates the user's emotional state in real time using an emotion analysis API (e.g., Microsoft® Azure® Emotion API). Based on this analysis data, it generates multiple hairstyle candidates that fit the user's emotions using a generative AI model (e.g., OpenAI® GPT-3®).

[0201] The generated hairstyles are prioritized and rearranged based on the user's emotional state and hair characteristics. For example, if the user indicates that they want to relax, the server will prioritize presenting styles that are appropriate for that emotion.

[0202] The presented style options are shown to beauty professionals via visual devices such as smart glasses, allowing them to perform treatments that take the user's emotional state into consideration. Smooth interaction between the user and the beauty professional leads to the provision of highly satisfying services.

[0203] For example, if a user appears tense upon arriving at the salon, the system might prioritize styles that promote relaxation. In this way, the system provides a hairstyle that best matches the user's preferences and emotional state.

[0204] An example of a prompt message to be used in a generative AI model is, "We have recognized from the user's facial expression that they want to relax, so please suggest a hairstyle that emphasizes relaxation."

[0205] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0206] Step 1:

[0207] The terminal provides an interface for the user to take a picture of their face and select their desired hairstyle. Based on the data entered by the user, the face image and hairstyle information are combined into a data packet and sent to the server. The input is the user's face image and style information, and the output is a data packet.

[0208] Step 2:

[0209] The server analyzes the received data packets. It processes the facial image using a facial recognition API and extracts the user's facial characteristics as digital data. This process yields facial feature point data. The input is a facial image, and the output is facial feature point data.

[0210] Step 3:

[0211] Furthermore, the server uses an emotion analysis API to analyze the user's emotional state from their facial image. This analysis determines the user's current emotional state and outputs it as emotion data. The input is a facial image, and the output is emotion data.

[0212] Step 4:

[0213] Based on the aforementioned facial feature point data and emotion data, the server utilizes a generative AI model to generate hairstyle candidates suitable for the user. In this process, prompts are used to provide input to the AI ​​model requesting the generation of specific styles, and a list of hairstyle candidates is output.

[0214] Step 5:

[0215] The server prioritizes and sorts the generated hairstyle candidates according to the user's emotional state. Based on emotional feedback, it adjusts the order of the styles and outputs a reorganized list of candidates. The input is hairstyle candidates, and the output is a reorganized list of hairstyles.

[0216] Step 6:

[0217] A rearranged list of hairstyles is presented to the user through a visual device, allowing for selection. The user chooses their desired style, and this selection is recorded. The input is the rearranged list of hairstyles, and the output is the user's final selection.

[0218] Step 7:

[0219] The final selected style information is provided to beauty professionals for reference during treatment. This information also includes treatment considerations based on the user's emotional state. The input is the user's final selection, and the output is the treatment information provided to beauty professionals.

[0220] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0221] Data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of data generation model 58 is ChatGPT (registered trademark) (Internet search).<URL: https: / / openai.com / blog / chatgpt> ), Gemini (registered trademark) (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0222] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart device 14.

[0223] [Second Embodiment]

[0224] Figure 3 shows an example of the configuration of the data processing system 210 according to the second embodiment.

[0225] As shown in Figure 3, the data processing system 210 includes a data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0226] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0227] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication interface 44. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, and camera 42 are also connected to the bus 52.

[0228] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0229] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0230] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0231] Figure 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Figure 4, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0232] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0233] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0234] In the smart glasses 214, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0235] Next, the identification processing performed by the identification processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0236] This invention relates to a method and apparatus for implementing an "AI haircut ordering system." The system generates an optimal hairstyle based on the user's facial image and desired hairstyle information, and presents it to the user in advance, thereby reducing the gap between the user's wishes and the actual finished look.

[0237] User actions

[0238] First, the user downloads and launches a dedicated application on their device. Using the application, the user takes a photo of their face and selects their desired hairstyle. This information is then used as input for the system.

[0239] Terminal processing

[0240] The device generates a data packet to send a facial image and desired hairstyle information to the server. The data packet includes the user's individual facial image and information about the style selected by the user. If the user's allergy information has been registered in advance, that information is also included.

[0241] Server Processing

[0242] The server analyzes the received data and uses an algorithm to analyze facial image features to extract the user's facial contours and hair characteristics. Next, a generative AI model is used to generate multiple hairstyle candidates based on the analyzed features. This generative AI model has been trained on past hairstyle datasets and the latest trend information, and provides the optimal candidate based on this.

[0243] The multiple style options generated from the analysis results are sent back to the terminal for user confirmation. The user can select the style they like best from the presented options. This selected style is recorded as the final order and provided to the beauty professional.

[0244] Specific example

[0245] For example, if a user wants a "short bob style," they log into the system, upload a photo of their face, search for "short bob," and select their favorite style. The system analyzes the facial image, generates several suitable bob style variations, and presents them to the user. The style B selected by the user is ultimately recorded and provided to a beauty professional along with related information.

[0246] This system allows users to see an image of the finished look beforehand, minimizing the gap between their expectations and the final result, and making their salon experience more satisfying.

[0247] The following describes the processing flow.

[0248] Step 1:

[0249] The user launches a smartphone application, follows the instructions to take a picture of their face, and searches for and selects their preferred hairstyle. The image and selected style information are recorded on the device.

[0250] Step 2:

[0251] The device generates facial images and hairstyle information as data packets and sends them to the server. The data may also include the user's allergy information.

[0252] Step 3:

[0253] The server analyzes the received data, examining the user's facial contours and hair texture from the facial image. A facial recognition algorithm is used to extract detailed hair characteristics.

[0254] Step 4:

[0255] The server runs an AI model and generates multiple hairstyle options that suit the user's characteristics based on the analysis results. This model takes trend information into account and suggests the optimal style.

[0256] Step 5:

[0257] The server sends the generated hairstyle options to the terminal.

[0258] Step 6:

[0259] The device presents the user with several style options it has received. The user reviews them and selects the style they like best.

[0260] Step 7:

[0261] The terminal records the style information selected by the user as the final order and sends it to a server for communication with beauty professionals.

[0262] Step 8:

[0263] The server provides beauty professionals with details on the selected style, the treatment method, and the necessary products. This ensures that the treatment at the salon runs smoothly.

[0264] (Example 1)

[0265] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0266] It is essential to reduce the gap between the hairstyle the user desires and the actual finished look, and to provide the optimal style that suits each individual's characteristics. Furthermore, it is necessary to perform safe and effective treatments based on the user's health information.

[0267] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0268] In this invention, the server includes means for analyzing facial images received from the user, desired hairstyle information, and registered health information; means for presenting multiple hairstyle candidates generated using the analysis results and a generation AI model; and means for providing the hairstyle information selected by the user to a specialist to support the provision of optimal service. This makes it possible to suggest a style suitable for the user and to provide safe and highly satisfying beauty services.

[0269] A "user" refers to an individual who uses the system to receive suggestions and make selections for their own hairstyle.

[0270] A "face image" is digital image data that includes the features of the user's face.

[0271] "Hair style information" refers to digital data and attribute information related to the hairstyle desired by the user.

[0272] "Health information" refers to information related to the user's health status and allergies.

[0273] "Analysis method" refers to a method of processing facial images and hairstyle information based on received data and extracting their characteristics.

[0274] A "generative AI model" is an artificial intelligence program that creates optimal hairstyle options for a user based on past data and trends.

[0275] "Presentation method" refers to the method of displaying the analyzed and generated hairstyle candidates to the user.

[0276] A "specialist" refers to a professional who possesses the skills and knowledge related to beauty and provides treatments and advice to users.

[0277] This section describes the embodiments for carrying out the invention. This system, an "AI haircut ordering system," utilizes smart devices and computer servers to facilitate end-user use. The overall system overview and operation are described in detail below.

[0278] Users access the platform by downloading a dedicated application to their smart device and launching it. Within the application, users take a photo of their face using the camera and select their desired hairstyle. This automatically sends the user's face image and hairstyle information to the system.

[0279] The device generates a data packet containing the captured facial image and the user's selected hairstyle information, and prepares it for transmission to the server. This data packet may also include pre-registered health information.

[0280] The server analyzes the received data packets. On the server, image analysis software (e.g., OpenCV or DeepFace) is used to extract the features of the face image. Also, based on the analysis data, a generative AI model (e.g., an AI program built with TensorFlow) is used to generate candidates for various hairstyles. This AI model has learned past data and the latest trends and can present an optimal style based on the user's preferences and characteristics.

[0281] The generated hairstyle candidates are sent to the user's terminal, and the user can select the one that matches their preference from multiple styles. This selected information is shared with beauty experts and relevant staff and used to provide optimal beauty services to the user.

[0282] As a specific example, after the user logs in to the app, they use the "Camera" function to upload their photo and then search for and select the "Short Bob" style. Based on this information, the server inputs a prompt sentence "Generate a Short Bob style that suits the features of this face" to the generative AI model to generate an appropriate style. [[ID=�10]]

[0283] The flow of the specific process in Example 1 will be described using FIG. 11.

[0284] Step 1:

[0285] The user launches a dedicated application on the smart device, takes a face image with the camera, and selects the desired hairstyle. The input includes the face image data taken by the user and the selected hairstyle information. Through the user's operation, this information is saved in the terminal.

[0286] Step 2:

[0287] The device generates a data packet containing saved facial images and hairstyle information, as well as health information previously registered by the user. This data packet is structured in an appropriate format (e.g., JSON format) and prepared for transmission to the server.

[0288] Step 3:

[0289] The server analyzes data packets received from the terminal. The input is data packets, and the facial images contained within them are processed using image analysis software (e.g., OpenCV) to extract the user's facial features. The output is a feature vector containing facial contours and hair texture information.

[0290] Step 4:

[0291] The server uses a generative AI model to generate hairstyle candidates based on extracted facial features and input hairstyle information. The input consists of feature vectors, hairstyle information, and a prompt message, such as "Generate hairstyles that match these facial features." The output includes image data of multiple hairstyle candidates.

[0292] Step 5:

[0293] The image data of hairstyle candidates generated on the server is sent back to the terminal for user confirmation. The terminal displays the candidate styles on the screen for the user. Based on this information, the user can select the most suitable hairstyle.

[0294] Step 6:

[0295] The hairstyle information selected by the user is recorded as the final order and provided to the stylist. The server transmits this information to the stylist in the appropriate format to help prepare the beauty service for the user.

[0296] (Application Example 1)

[0297] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0298] Traditionally, hairstyle simulations and selections have often relied on static images, leading to discrepancies between the simulated look and the actual finished style. This makes it difficult for users to achieve satisfactory results in salons, resulting in dissatisfaction. Furthermore, there is a lack of sufficient means for users to visually try out suitable styles in real time.

[0299] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0300] In this invention, the server includes means for analyzing facial information received from the user and the selected appearance information; means for presenting a plurality of appearance candidates generated based on the analysis results; means for providing the selected appearance information; and means for displaying the change in appearance using the user's head image. This allows the user to visually try out different styles in real time using a smart device and select the appearance that best matches their preferences.

[0301] "Facial information received from the user" refers to image data of the user's face, which is used to analyze facial features.

[0302] "Selected appearance information" refers to information about the appearance and style desired by the user, and is data used to select appearances.

[0303] "Means of analysis" refers to the processes and technologies used to determine the suitability of a user's characteristics and style based on the received data.

[0304] "Means for presenting multiple candidate appearances" refers to a method that has the function of visually showing the user multiple styles generated based on the analysis results.

[0305] The means for providing the selected appearance information refers to a system that stores the data of the appearance selected by the user and provides it for external assistance if necessary.

[0306] The device for displaying appearance changes using the user's head image is a device that projects appearance changes onto the user's real head image and provides a visual simulation.

[0307] This invention provides an interactive appearance simulation system. The server performs appearance simulation using the face image received from the user's smart device and the selected appearance information. The server uses image processing software such as OpenCV and TensorFlow for executing a generative AI model to analyze the face image and extract the user's individual features. Thereby, it is possible to identify characteristics such as the shape and dimensions of the user's face and hair texture.

[0308] Based on the identified features, the server uses a trained AI model to generate multiple appearance candidates that match those features. These candidates reflect the latest trend information and provide the user with diverse options.

[0309] The terminal displays the generated appearance candidates in real time on the display of the smart glasses. The user can visually try various appearances through the glasses and can easily switch the appearance using voice or a gaze tracking sensor.

[0310] As a specific example, when the user wants to try a "casual appearance", a prompt sentence is used to instruct the AI model to "generate multiple variations of a casual style from the user's face image and display them in real time". According to this prompt sentence, the AI presents the optimal appearance candidates to the user, and the user can confirm and select the desired appearance. With this system, the user can easily try appearances in a physical store and choose the one they like the most, thus providing a highly satisfactory experience.

[0311] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0312] Step 1:

[0313] The user takes a facial image using a smart device and selects desired appearance information. This information is entered into the device. The device then generates a data packet containing the facial image and selected appearance information and sends it to the server.

[0314] Step 2:

[0315] The server extracts the user's face image and appearance information from the received data packets. The server uses OpenCV to analyze the face image and identify facial features. Specifically, it detects the shape, dimensions, and hair texture of the face. The output of this process is a numerical representation of the facial features.

[0316] Step 3:

[0317] The server inputs the obtained facial features into a generating AI model and generates appearance candidates based on the prompt "Generate multiple appearance variations from the user's face image." The AI ​​model considers pre-trained trend information and outputs the most suitable appearance.

[0318] Step 4:

[0319] The generated appearance options are sent to the terminal by the server. The terminal displays the appearance changes in real time on the smart glasses' display. The user can visually confirm the appearance through the glasses and switch between options using gaze or voice commands.

[0320] Step 5:

[0321] The user's final selected appearance information is recorded on the device and uploaded to the server. This recorded information is then provided to the relevant service providers and used for future service improvements and personalized support.

[0322] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0323] This invention relates to an "AI haircut ordering system" that combines a "emotion engine" that acquires a user's facial image and desired hairstyle information and recognizes the user's emotions. The system generates hairstyle candidates that take into account the user's hair characteristics and emotions, and provides them to beauty professionals, thereby providing a service that fits the user's wishes and emotions.

[0324] User actions

[0325] The user downloads and launches a smartphone application. After launching, they use their device to take a photo of their face and select their desired hairstyle. The application uses this information as system input.

[0326] Terminal processing

[0327] The device forms a data packet containing the captured facial image and the user's selected hairstyle information, and sends it to the server. If the user's allergy information has been registered in advance, that information will also be included in the data.

[0328] Processing by the emotion engine

[0329] The server uses an emotion engine to analyze received facial images, determining the user's emotional state from their facial expressions. This emotion recognition helps evaluate whether the style chosen by the user fits their emotional state.

[0330] Server Processing

[0331] Taking into account the user's emotions recognized by the emotion engine, the server uses a generative AI model to generate multiple hairstyle options that reflect the user's characteristics. The order in which the generated styles are presented may be adjusted based on the emotions. Information suggesting precautions for beauty professionals during treatment can also be generated.

[0332] Presenting the style

[0333] The terminal presents the user with multiple style options received from the server. The user selects their favorite style from the presented options, which is then recorded as the final order and provided to the beauty professional.

[0334] Specific example

[0335] For example, if a user requests "long curls," but the emotion engine recognizes from the user's current facial expression that they want to "relax," the system will prioritize suggesting styles that emphasize "relaxation." The style chosen by the user is recorded as the final order, and during the treatment, the beauty professional is provided with notes based on the emotion recognition.

[0336] This system allows us to provide hairstyle services that suit both the user's preferences and emotional state, thereby improving the user experience.

[0337] The following describes the processing flow.

[0338] Step 1:

[0339] The user launches a smartphone application, follows the instructions to take a picture of their face, and selects their desired hairstyle. The face image and selected style information are then entered into the device.

[0340] Step 2:

[0341] The device forms a data packet containing the captured facial image and selected hairstyle information, and sends it to the server. The data may also include the user's allergy information.

[0342] Step 3:

[0343] The server analyzes the received facial image using an emotion engine, recognizing the user's emotional state from their facial expressions. This allows for responses tailored to the user's emotions, such as a style that makes them feel safe and relaxed, or a style that energizes them.

[0344] Step 4:

[0345] The server analyzes hair characteristics based on the facial image and selected hairstyle information, and uses a generative AI model to generate multiple hairstyle candidates suitable for the user. The priority of the style candidates is adjusted based on the recognized emotional state.

[0346] Step 5:

[0347] The server sends the generated hairstyle suggestions and their detailed information to the terminal. This includes style images, key cutting points, recommended products, and emotionally-driven suggestions.

[0348] Step 6:

[0349] The device presents the user with multiple style options it has received. The user selects the style that seems to fit best from the presented options. This selection is recorded as the final order.

[0350] Step 7:

[0351] The selected style information is sent again from the terminal to the server and provided to the beauty professional as the final decision.

[0352] Step 8:

[0353] The server provides beauty professionals with detailed information, including the selected style and considerations based on the user's emotional state to be taken into account when performing the treatment. This ensures that the treatment aligns with the user's emotions, leading to increased satisfaction.

[0354] (Example 2)

[0355] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0356] Traditionally, there have been no systems that consider the user's emotional state when selecting a hairstyle, resulting in users not achieving their desired level of satisfaction. Furthermore, in order to suggest an appropriate style, it is necessary to accurately reflect information about each individual's head characteristics and health, but previous systems have not adequately addressed this issue.

[0357] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0358] In this invention, the server includes means for analyzing a facial image received from the user and selected style information, means for recognizing the user's emotional state from the facial image using an emotion engine, and means for a generating AI model to generate and present multiple style candidates based on the analysis results and emotional state. This makes it possible to suggest styles suitable for individual characteristics while taking into account the user's emotional state, and further select appropriate products and services based on health information.

[0359] A "user" refers to an individual who uses the system to provide their facial image and style information in order to receive beauty services.

[0360] "Facial images" refer to image data captured in order to analyze a user's facial expressions and head characteristics.

[0361] "Style information" refers to information about the hairstyle desired by the user, including the type and characteristics of the selected style.

[0362] An "emotion engine" refers to software or algorithms that analyze a user's facial image and recognize their emotional state from their facial expressions.

[0363] A "generative AI model" refers to artificial intelligence technology that generates optimal style candidates based on the user's characteristics and emotional state.

[0364] "Hairstyle options" refers to the multiple hairstyle choices suggested to the user, from which the user selects the one that best suits them.

[0365] "Health-related information" refers to allergy information and other health status data provided by users, which is used to select appropriate products and services.

[0366] A "beauty professional" refers to a specialized service provider who performs hair styling treatments for users.

[0367] The specific operation of the system in an embodiment for carrying out this invention is shown below.

[0368] First, the user launches a dedicated application installed on their mobile device. Using the application, the user can take a photo of their face and input their desired style information. This face photo and style information are stored on the device as data packets. If necessary, the user can register health information in the application and record allergies and special precautions.

[0369] The terminal sends the generated data packets to the server. This transmission takes place via the internet, ensuring high speed and security. The server is equipped with advanced image analysis software and an emotion engine, which evaluates the user's emotional state from the received facial images. This emotion engine is based on algorithms that analyze the user's facial features and identify emotions in real time.

[0370] Furthermore, the server utilizes a generative AI model to generate multiple style options that suit the user's characteristics and emotions. These generated style options are then presented in a prioritized manner based on the user's emotional state. For example, if the server determines that the user desires "long curls" but also wants to relax, it will prioritize suggesting styles that evoke a strong sense of relaxation.

[0371] Style information transmitted from the server is displayed on the terminal. The user can view style options on the screen and select their preferred one. The selected style is recorded as the final choice and, when provided to a beauty professional, is accompanied by treatment advice based on the emotional criteria of the suggested style.

[0372] Throughout this entire process, users can receive personalized beauty services tailored to their emotional state and preferences.

[0373] An example of a prompt message might be, "Suggest a hairstyle suitable for a user who wants to relax." This allows the system to generate and provide a style that incorporates the user's emotions and desires.

[0374] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0375] Step 1:

[0376] The user launches an application installed on their mobile device and takes a photo of their face. The application provides an input screen where the user can select their desired hairstyle. This input process yields both the facial image data and the selected style information.

[0377] Step 2:

[0378] The terminal composes the input facial image data and style information into a data packet. This data packet also includes health information registered by the user, if necessary. The terminal prepares to transmit this data packet to the server via the network connection.

[0379] Step 3:

[0380] The server receives data packets over the network. Based on the received data, it processes the facial image using image analysis software to extract the user's facial features. This outputs facial feature quantities from the image data for input into the emotion engine.

[0381] Step 4:

[0382] The server inputs the extracted facial features into the emotion engine. The emotion engine uses these features to recognize the user's emotional state. This process allows for the recognition of emotional states such as "joy" and "relaxation."

[0383] Step 5:

[0384] The server inputs emotional state and style information into a generative AI model, which generates style candidates suitable for the user's characteristics and emotional state. The generative AI model uses machine learning techniques to output multiple style candidates from this data.

[0385] Step 6:

[0386] The server prioritizes the generated style candidates based on the user's emotional state. For example, if the user is perceived as wanting to relax, styles that emphasize "relaxation" will be prioritized. This prioritized style information is then output.

[0387] Step 7:

[0388] The terminal presents the user with style options received from the server. The user can view multiple style options on the screen and select the one they like best. This selection then provides the final selected style information.

[0389] Step 8:

[0390] The terminal sends the user's selected style information to the server as the final order. The server provides the beauty professional with treatment considerations based on the selected style and emotions, helping them to provide a service that matches the user's wishes and emotions.

[0391] (Application Example 2)

[0392] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0393] Conventional hairstyle suggestion systems have limitations in improving the user experience because they make suggestions without considering the user's emotional state. Furthermore, it was difficult to suggest the most suitable hairstyle to the customer in real time. These challenges made it difficult to provide services and customer care that would satisfy users.

[0394] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0395] In this invention, the server includes means for analyzing a facial image received from the user and information on the desired hairstyle, means for analyzing the user's emotional state in real time using a visual device, and means for adjusting suggested hairstyles based on the user's emotional state. This makes it possible to suggest hairstyles that are suitable for the user's emotions.

[0396] "Means for analyzing facial images and desired hairstyle information received from a user" refers to a processing device that uses digital technology to analyze facial image data and information about desired hairstyles obtained from a user, and extracts style candidates based on the results.

[0397] "Means for presenting multiple hairstyle options generated based on analysis results" refers to an interface for visually or electronically suggesting multiple hairstyles generated from the analysis results to the user.

[0398] "Means for providing selected hairstyle information to beauty professionals" refers to an information provision device that transmits information about the hairstyle selected by the user to beauty professionals, enabling them to perform treatments based on that information.

[0399] "Means for analyzing a user's emotional state in real time using visual devices" refers to a technology that uses visual devices such as cameras to instantly determine emotions from a user's facial expressions and other visual information, and then analyzes that information.

[0400] "Means for adjusting suggested hairstyles based on the user's emotional state" refers to an algorithm or system that adjusts the priority and content of suggested hairstyles according to the results of emotion analysis, thereby providing the user with the most suitable option based on their mood and emotions.

[0401] In implementing this invention, the user first accesses the system using a terminal device such as a smartphone or tablet. The user takes a picture of their face and selects their desired hairstyle. This information is digitized on the terminal and transmitted to the server. The server utilizes image recognition technology and sentiment analysis software to analyze the received face image and hairstyle information.

[0402] Specifically, the server analyzes the user's face using a facial recognition API (e.g., AWS Rekognition) and evaluates the user's emotional state in real time using an emotion analysis API (e.g., Microsoft Azure Emotion API). Based on this analysis data, it uses a generative AI model (e.g., OpenAI GPT-3) to generate multiple hairstyle candidates that fit the user's emotions.

[0403] The generated hairstyles are prioritized and rearranged based on the user's emotional state and hair characteristics. For example, if the user indicates that they want to relax, the server will prioritize presenting styles that are appropriate for that emotion.

[0404] The presented style options are shown to beauty professionals via visual devices such as smart glasses, allowing them to perform treatments that take the user's emotional state into consideration. Smooth interaction between the user and the beauty professional leads to the provision of highly satisfying services.

[0405] For example, if a user appears tense upon arriving at the salon, the system might prioritize styles that promote relaxation. In this way, the system provides a hairstyle that best matches the user's preferences and emotional state.

[0406] An example of a prompt message to be used in a generative AI model is, "We have recognized from the user's facial expression that they want to relax, so please suggest a hairstyle that emphasizes relaxation."

[0407] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0408] Step 1:

[0409] The terminal provides an interface for the user to take a picture of their face and select their desired hairstyle. Based on the data entered by the user, the face image and hairstyle information are combined into a data packet and sent to the server. The input is the user's face image and style information, and the output is a data packet.

[0410] Step 2:

[0411] The server analyzes the received data packets. It processes the facial image using a facial recognition API and extracts the user's facial characteristics as digital data. This process yields facial feature point data. The input is a facial image, and the output is facial feature point data.

[0412] Step 3:

[0413] Furthermore, the server uses an emotion analysis API to analyze the user's emotional state from their facial image. This analysis determines the user's current emotional state and outputs it as emotion data. The input is a facial image, and the output is emotion data.

[0414] Step 4:

[0415] Based on the aforementioned facial feature point data and emotion data, the server utilizes a generative AI model to generate hairstyle candidates suitable for the user. In this process, prompts are used to provide input to the AI ​​model requesting the generation of specific styles, and a list of hairstyle candidates is output.

[0416] Step 5:

[0417] The server prioritizes and sorts the generated hairstyle candidates according to the user's emotional state. Based on emotional feedback, it adjusts the order of the styles and outputs a reorganized list of candidates. The input is hairstyle candidates, and the output is a reorganized list of hairstyles.

[0418] Step 6:

[0419] A rearranged list of hairstyles is presented to the user through a visual device, allowing for selection. The user chooses their desired style, and this selection is recorded. The input is the rearranged list of hairstyles, and the output is the user's final selection.

[0420] Step 7:

[0421] The final selected style information is provided to beauty professionals for reference during treatment. This information also includes treatment considerations based on the user's emotional state. The input is the user's final selection, and the output is the treatment information provided to beauty professionals.

[0422] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0423] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet Search).<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0424] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart glasses 214.

[0425] [Third Embodiment]

[0426] Figure 5 shows an example of the configuration of the data processing system 310 according to the third embodiment.

[0427] As shown in Figure 5, the data processing system 310 includes a data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[0428] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0429] The headset terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a display 343. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and display 343 are also connected to the bus 52.

[0430] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0431] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0432] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0433] Figure 6 shows an example of the main functions of the data processing device 12 and the headset terminal 314. As shown in Figure 6, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0434] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0435] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0436] In the headset terminal 314, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0437] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the headset terminal 314 will be referred to as the "terminal".

[0438] This invention relates to a method and apparatus for implementing an "AI haircut ordering system." The system generates an optimal hairstyle based on the user's facial image and desired hairstyle information, and presents it to the user in advance, thereby reducing the gap between the user's wishes and the actual finished look.

[0439] User actions

[0440] First, the user downloads and launches a dedicated application on their device. Using the application, the user takes a photo of their face and selects their desired hairstyle. This information is then used as input for the system.

[0441] Terminal processing

[0442] The device generates a data packet to send a facial image and desired hairstyle information to the server. The data packet includes the user's individual facial image and information about the style selected by the user. If the user's allergy information has been registered in advance, that information is also included.

[0443] Server Processing

[0444] The server analyzes the received data and uses an algorithm to analyze facial image features to extract the user's facial contours and hair characteristics. Next, a generative AI model is used to generate multiple hairstyle candidates based on the analyzed features. This generative AI model has been trained on past hairstyle datasets and the latest trend information, and provides the optimal candidate based on this.

[0445] The multiple style options generated from the analysis results are sent back to the terminal for user confirmation. The user can select the style they like best from the presented options. This selected style is recorded as the final order and provided to the beauty professional.

[0446] Specific example

[0447] For example, if a user wants a "short bob style," they log into the system, upload a photo of their face, search for "short bob," and select their favorite style. The system analyzes the facial image, generates several suitable bob style variations, and presents them to the user. The style B selected by the user is ultimately recorded and provided to a beauty professional along with related information.

[0448] This system allows users to see an image of the finished look beforehand, minimizing the gap between their expectations and the final result, and making their salon experience more satisfying.

[0449] The following describes the processing flow.

[0450] Step 1:

[0451] The user launches a smartphone application, follows the instructions to take a picture of their face, and searches for and selects their preferred hairstyle. The image and selected style information are recorded on the device.

[0452] Step 2:

[0453] The device generates facial images and hairstyle information as data packets and sends them to the server. The data may also include the user's allergy information.

[0454] Step 3:

[0455] The server analyzes the received data, examining the user's facial contours and hair texture from the facial image. A facial recognition algorithm is used to extract detailed hair characteristics.

[0456] Step 4:

[0457] The server runs an AI model and generates multiple hairstyle options that suit the user's characteristics based on the analysis results. This model takes trend information into account and suggests the optimal style.

[0458] Step 5:

[0459] The server sends the generated hairstyle options to the terminal.

[0460] Step 6:

[0461] The device presents the user with several style options it has received. The user reviews them and selects the style they like best.

[0462] Step 7:

[0463] The terminal records the style information selected by the user as the final order and sends it to a server for communication with beauty professionals.

[0464] Step 8:

[0465] The server provides beauty professionals with details on the selected style, the treatment method, and the necessary products. This ensures that the treatment at the salon runs smoothly.

[0466] (Example 1)

[0467] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0468] It is essential to reduce the gap between the hairstyle the user desires and the actual finished look, and to provide the optimal style that suits each individual's characteristics. Furthermore, it is necessary to perform safe and effective treatments based on the user's health information.

[0469] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0470] In this invention, the server includes means for analyzing facial images received from the user, desired hairstyle information, and registered health information; means for presenting multiple hairstyle candidates generated using the analysis results and a generation AI model; and means for providing the hairstyle information selected by the user to a specialist to support the provision of optimal service. This makes it possible to suggest a style suitable for the user and to provide safe and highly satisfying beauty services.

[0471] A "user" refers to an individual who uses the system to receive suggestions and make selections for their own hairstyle.

[0472] A "face image" is digital image data that includes the features of the user's face.

[0473] "Hair style information" refers to digital data and attribute information related to the hairstyle desired by the user.

[0474] "Health information" refers to information related to the user's health status and allergies.

[0475] "Analysis method" refers to a method of processing facial images and hairstyle information based on received data and extracting their characteristics.

[0476] A "generative AI model" is an artificial intelligence program that creates optimal hairstyle options for a user based on past data and trends.

[0477] "Presentation method" refers to the method of displaying the analyzed and generated hairstyle candidates to the user.

[0478] A "specialist" refers to a professional who possesses the skills and knowledge related to beauty and provides treatments and advice to users.

[0479] This section describes the embodiments for carrying out the invention. This system, an "AI haircut ordering system," utilizes smart devices and computer servers to facilitate end-user use. The overall system overview and operation are described in detail below.

[0480] Users access the platform by downloading a dedicated application to their smart device and launching it. Within the application, users take a photo of their face using the camera and select their desired hairstyle. This automatically sends the user's face image and hairstyle information to the system.

[0481] The device generates a data packet containing the captured facial image and the user's selected hairstyle information, and prepares it for transmission to the server. This data packet may also include pre-registered health information.

[0482] The server analyzes the received data packets. On the server, it extracts facial features using image analysis software (e.g., OpenCV or DeepFace). Based on the analyzed data, it generates various hairstyle candidates using a generative AI model (e.g., an AI program built with TensorFlow). This AI model has been trained on historical data and the latest trends, and can suggest the optimal style based on the user's preferences and characteristics.

[0483] The generated hairstyle options are sent to the user's device, allowing the user to select the style that best suits their preferences from among several options. This selected information is shared with beauty professionals and related staff and used to provide the user with the most suitable beauty services.

[0484] As a concrete example, after a user logs into the app, they upload a photo of themselves using the "camera" function, and then search for and select a "short bob" style. Based on this information, the server inputs a prompt message to the AI ​​model saying, "Generate a short bob style that suits these facial features," and generates an appropriate style.

[0485] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0486] Step 1:

[0487] The user launches a dedicated application on their smart device, takes a picture of their face with the camera, and selects their desired hairstyle. The input includes the facial image data the user has captured and the selected hairstyle information. This information is saved on the device based on the user's actions.

[0488] Step 2:

[0489] The device generates a data packet containing saved facial images and hairstyle information, as well as health information previously registered by the user. This data packet is structured in an appropriate format (e.g., JSON format) and prepared for transmission to the server.

[0490] Step 3:

[0491] The server analyzes data packets received from the terminal. The input is data packets, and the facial images contained within them are processed using image analysis software (e.g., OpenCV) to extract the user's facial features. The output is a feature vector containing facial contours and hair texture information.

[0492] Step 4:

[0493] The server uses a generative AI model to generate hairstyle candidates based on extracted facial features and input hairstyle information. The input consists of feature vectors, hairstyle information, and a prompt message, such as "Generate hairstyles that match these facial features." The output includes image data of multiple hairstyle candidates.

[0494] Step 5:

[0495] The image data of hairstyle candidates generated on the server is sent back to the terminal for user confirmation. The terminal displays the candidate styles on the screen for the user. Based on this information, the user can select the most suitable hairstyle.

[0496] Step 6:

[0497] The hairstyle information selected by the user is recorded as the final order and provided to the stylist. The server transmits this information to the stylist in the appropriate format to help prepare the beauty service for the user.

[0498] (Application Example 1)

[0499] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0500] Traditionally, hairstyle simulations and selections have often relied on static images, leading to discrepancies between the simulated look and the actual finished style. This makes it difficult for users to achieve satisfactory results in salons, resulting in dissatisfaction. Furthermore, there is a lack of sufficient means for users to visually try out suitable styles in real time.

[0501] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0502] In this invention, the server includes means for analyzing facial information received from the user and the selected appearance information; means for presenting a plurality of appearance candidates generated based on the analysis results; means for providing the selected appearance information; and means for displaying the change in appearance using the user's head image. This allows the user to visually try out different styles in real time using a smart device and select the appearance that best matches their preferences.

[0503] "Facial information received from the user" refers to image data of the user's face, which is used to analyze facial features.

[0504] "Selected appearance information" refers to information about the appearance and style desired by the user, and is data used to select appearances.

[0505] "Means of analysis" refers to the processes and technologies used to determine the suitability of a user's characteristics and style based on the received data.

[0506] "Means for presenting multiple candidate appearances" refers to a method that has the function of visually showing the user multiple styles generated based on the analysis results.

[0507] "Means of providing selected appearance information" refers to a system that stores data on the appearance selected by the user and provides it to external assistance as needed.

[0508] A "device that displays changes in appearance using a user's head image" is a device that projects changes in appearance onto a real image of the user's head, providing a visual simulation.

[0509] This invention provides an interactive appearance simulation system. The server performs an appearance simulation using a face image received from the user's smart device and selected appearance information. The server analyzes the face image using image processing software such as OpenCV and TensorFlow for executing generative AI models, and extracts individual user features. This makes it possible to identify characteristics such as the shape and dimensions of the user's face and hair texture.

[0510] Based on the identified features, the server uses a pre-trained AI model to generate multiple appearance candidates that match those features. These candidates reflect the latest trends, providing users with a variety of choices.

[0511] The device displays generated appearance options in real time on the smart glasses' display. Users can visually try out various appearances through the glasses and easily switch between them using voice and eye-tracking sensors.

[0512] For example, if a user wants to try a "casual look," they can use a prompt to instruct the AI ​​model to "generate multiple variations of casual styles from the user's facial image and display them in real time." Following this prompt, the AI ​​will present the user with the most suitable look options, which the user can then review and select their preferred look. This system allows users to easily try out different looks in a physical store and choose the one they like best, resulting in a highly satisfying experience.

[0513] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0514] Step 1:

[0515] The user takes a facial image using a smart device and selects desired appearance information. This information is entered into the device. The device then generates a data packet containing the facial image and selected appearance information and sends it to the server.

[0516] Step 2:

[0517] The server extracts the user's face image and appearance information from the received data packets. The server uses OpenCV to analyze the face image and identify facial features. Specifically, it detects the shape, dimensions, and hair texture of the face. The output of this process is a numerical representation of the facial features.

[0518] Step 3:

[0519] The server inputs the obtained facial features into a generating AI model and generates appearance candidates based on the prompt "Generate multiple appearance variations from the user's face image." The AI ​​model considers pre-trained trend information and outputs the most suitable appearance.

[0520] Step 4:

[0521] The generated appearance options are sent to the terminal by the server. The terminal displays the appearance changes in real time on the smart glasses' display. The user can visually confirm the appearance through the glasses and switch between options using gaze or voice commands.

[0522] Step 5:

[0523] The user's final selected appearance information is recorded on the device and uploaded to the server. This recorded information is then provided to the relevant service providers and used for future service improvements and personalized support.

[0524] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0525] This invention relates to an "AI haircut ordering system" that combines a "emotion engine" that acquires a user's facial image and desired hairstyle information and recognizes the user's emotions. The system generates hairstyle candidates that take into account the user's hair characteristics and emotions, and provides them to beauty professionals, thereby providing a service that fits the user's wishes and emotions.

[0526] User actions

[0527] The user downloads and launches a smartphone application. After launching, they use their device to take a photo of their face and select their desired hairstyle. The application uses this information as system input.

[0528] Terminal processing

[0529] The device forms a data packet containing the captured facial image and the user's selected hairstyle information, and sends it to the server. If the user's allergy information has been registered in advance, that information will also be included in the data.

[0530] Processing by the emotion engine

[0531] The server uses an emotion engine to analyze received facial images, determining the user's emotional state from their facial expressions. This emotion recognition helps evaluate whether the style chosen by the user fits their emotional state.

[0532] Server Processing

[0533] Taking into account the user's emotions recognized by the emotion engine, the server uses a generative AI model to generate multiple hairstyle options that reflect the user's characteristics. The order in which the generated styles are presented may be adjusted based on the emotions. Information suggesting precautions for beauty professionals during treatment can also be generated.

[0534] Presenting the style

[0535] The terminal presents the user with multiple style options received from the server. The user selects their favorite style from the presented options, which is then recorded as the final order and provided to the beauty professional.

[0536] Specific example

[0537] For example, if a user requests "long curls," but the emotion engine recognizes from the user's current facial expression that they want to "relax," the system will prioritize suggesting styles that emphasize "relaxation." The style chosen by the user is recorded as the final order, and during the treatment, the beauty professional is provided with notes based on the emotion recognition.

[0538] This system allows us to provide hairstyle services that suit both the user's preferences and emotional state, thereby improving the user experience.

[0539] The following describes the processing flow.

[0540] Step 1:

[0541] The user launches a smartphone application, follows the instructions to take a picture of their face, and selects their desired hairstyle. The face image and selected style information are then entered into the device.

[0542] Step 2:

[0543] The device forms a data packet containing the captured facial image and selected hairstyle information, and sends it to the server. The data may also include the user's allergy information.

[0544] Step 3:

[0545] The server analyzes the received facial image using an emotion engine, recognizing the user's emotional state from their facial expressions. This allows for responses tailored to the user's emotions, such as a style that makes them feel safe and relaxed, or a style that energizes them.

[0546] Step 4:

[0547] The server analyzes hair characteristics based on the facial image and selected hairstyle information, and uses a generative AI model to generate multiple hairstyle candidates suitable for the user. The priority of the style candidates is adjusted based on the recognized emotional state.

[0548] Step 5:

[0549] The server sends the generated hairstyle suggestions and their detailed information to the terminal. This includes style images, key cutting points, recommended products, and emotionally-driven suggestions.

[0550] Step 6:

[0551] The device presents the user with multiple style options it has received. The user selects the style that seems to fit best from the presented options. This selection is recorded as the final order.

[0552] Step 7:

[0553] The selected style information is sent again from the terminal to the server and provided to the beauty professional as the final decision.

[0554] Step 8:

[0555] The server provides beauty professionals with detailed information, including the selected style and considerations based on the user's emotional state to be taken into account when performing the treatment. This ensures that the treatment aligns with the user's emotions, leading to increased satisfaction.

[0556] (Example 2)

[0557] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0558] Traditionally, there have been no systems that consider the user's emotional state when selecting a hairstyle, resulting in users not achieving their desired level of satisfaction. Furthermore, in order to suggest an appropriate style, it is necessary to accurately reflect information about each individual's head characteristics and health, but previous systems have not adequately addressed this issue.

[0559] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0560] In this invention, the server includes means for analyzing a facial image received from the user and selected style information, means for recognizing the user's emotional state from the facial image using an emotion engine, and means for a generating AI model to generate and present multiple style candidates based on the analysis results and emotional state. This makes it possible to suggest styles suitable for individual characteristics while taking into account the user's emotional state, and further select appropriate products and services based on health information.

[0561] A "user" refers to an individual who uses the system to provide their facial image and style information in order to receive beauty services.

[0562] "Facial images" refer to image data captured in order to analyze a user's facial expressions and head characteristics.

[0563] "Style information" refers to information about the hairstyle desired by the user, including the type and characteristics of the selected style.

[0564] An "emotion engine" refers to software or algorithms that analyze a user's facial image and recognize their emotional state from their facial expressions.

[0565] A "generative AI model" refers to artificial intelligence technology that generates optimal style candidates based on the user's characteristics and emotional state.

[0566] "Hairstyle options" refers to the multiple hairstyle choices suggested to the user, from which the user selects the one that best suits them.

[0567] "Health-related information" refers to allergy information and other health status data provided by users, which is used to select appropriate products and services.

[0568] A "beauty professional" refers to a specialized service provider who performs hair styling treatments for users.

[0569] The specific operation of the system in an embodiment for carrying out this invention is shown below.

[0570] First, the user launches a dedicated application installed on their mobile device. Using the application, the user can take a photo of their face and input their desired style information. This face photo and style information are stored on the device as data packets. If necessary, the user can register health information in the application and record allergies and special precautions.

[0571] The terminal sends the generated data packets to the server. This transmission takes place via the internet, ensuring high speed and security. The server is equipped with advanced image analysis software and an emotion engine, which evaluates the user's emotional state from the received facial images. This emotion engine is based on algorithms that analyze the user's facial features and identify emotions in real time.

[0572] Furthermore, the server utilizes a generative AI model to generate multiple style options that suit the user's characteristics and emotions. These generated style options are then presented in a prioritized manner based on the user's emotional state. For example, if the server determines that the user desires "long curls" but also wants to relax, it will prioritize suggesting styles that evoke a strong sense of relaxation.

[0573] Style information transmitted from the server is displayed on the terminal. The user can view style options on the screen and select their preferred one. The selected style is recorded as the final choice and, when provided to a beauty professional, is accompanied by treatment advice based on the emotional criteria of the suggested style.

[0574] Throughout this entire process, users can receive personalized beauty services tailored to their emotional state and preferences.

[0575] An example of a prompt message might be, "Suggest a hairstyle suitable for a user who wants to relax." This allows the system to generate and provide a style that incorporates the user's emotions and desires.

[0576] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0577] Step 1:

[0578] The user launches an application installed on their mobile device and takes a photo of their face. The application provides an input screen where the user can select their desired hairstyle. This input process yields both the facial image data and the selected style information.

[0579] Step 2:

[0580] The terminal composes the input facial image data and style information into a data packet. This data packet also includes health information registered by the user, if necessary. The terminal prepares to transmit this data packet to the server via the network connection.

[0581] Step 3:

[0582] The server receives data packets over the network. Based on the received data, it processes the facial image using image analysis software to extract the user's facial features. This outputs facial feature quantities from the image data for input into the emotion engine.

[0583] Step 4:

[0584] The server inputs the extracted facial features into the emotion engine. The emotion engine uses these features to recognize the user's emotional state. This process allows for the recognition of emotional states such as "joy" and "relaxation."

[0585] Step 5:

[0586] The server inputs emotional state and style information into a generative AI model, which generates style candidates suitable for the user's characteristics and emotional state. The generative AI model uses machine learning techniques to output multiple style candidates from this data.

[0587] Step 6:

[0588] The server prioritizes the generated style candidates based on the user's emotional state. For example, if the user is perceived as wanting to relax, styles that emphasize "relaxation" will be prioritized. This prioritized style information is then output.

[0589] Step 7:

[0590] The terminal presents the user with style options received from the server. The user can view multiple style options on the screen and select the one they like best. This selection then provides the final selected style information.

[0591] Step 8:

[0592] The terminal sends the user's selected style information to the server as the final order. The server provides the beauty professional with treatment considerations based on the selected style and emotions, helping them to provide a service that matches the user's wishes and emotions.

[0593] (Application Example 2)

[0594] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0595] Conventional hairstyle suggestion systems have limitations in improving the user experience because they make suggestions without considering the user's emotional state. Furthermore, it was difficult to suggest the most suitable hairstyle to the customer in real time. These challenges made it difficult to provide services and customer care that would satisfy users.

[0596] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0597] In this invention, the server includes means for analyzing a facial image received from the user and information on the desired hairstyle, means for analyzing the user's emotional state in real time using a visual device, and means for adjusting suggested hairstyles based on the user's emotional state. This makes it possible to suggest hairstyles that are suitable for the user's emotions.

[0598] "Means for analyzing facial images and desired hairstyle information received from a user" refers to a processing device that uses digital technology to analyze facial image data and information about desired hairstyles obtained from a user, and extracts style candidates based on the results.

[0599] "Means for presenting multiple hairstyle options generated based on analysis results" refers to an interface for visually or electronically suggesting multiple hairstyles generated from the analysis results to the user.

[0600] "Means for providing selected hairstyle information to beauty professionals" refers to an information provision device that transmits information about the hairstyle selected by the user to beauty professionals, enabling them to perform treatments based on that information.

[0601] "Means for analyzing a user's emotional state in real time using visual devices" refers to a technology that uses visual devices such as cameras to instantly determine emotions from a user's facial expressions and other visual information, and then analyzes that information.

[0602] "Means for adjusting suggested hairstyles based on the user's emotional state" refers to an algorithm or system that adjusts the priority and content of suggested hairstyles according to the results of emotion analysis, thereby providing the user with the most suitable option based on their mood and emotions.

[0603] In implementing this invention, the user first accesses the system using a terminal device such as a smartphone or tablet. The user takes a picture of their face and selects their desired hairstyle. This information is digitized on the terminal and transmitted to the server. The server utilizes image recognition technology and sentiment analysis software to analyze the received face image and hairstyle information.

[0604] Specifically, the server analyzes the user's face using a facial recognition API (e.g., AWS Rekognition) and evaluates the user's emotional state in real time using an emotion analysis API (e.g., Microsoft Azure Emotion API). Based on this analysis data, it uses a generative AI model (e.g., OpenAI GPT-3) to generate multiple hairstyle candidates that fit the user's emotions.

[0605] The generated hairstyles are prioritized and rearranged based on the user's emotional state and hair characteristics. For example, if the user indicates that they want to relax, the server will prioritize presenting styles that are appropriate for that emotion.

[0606] The presented style options are shown to beauty professionals via visual devices such as smart glasses, allowing them to perform treatments that take the user's emotional state into consideration. Smooth interaction between the user and the beauty professional leads to the provision of highly satisfying services.

[0607] For example, if a user appears tense upon arriving at the salon, the system might prioritize styles that promote relaxation. In this way, the system provides a hairstyle that best matches the user's preferences and emotional state.

[0608] An example of a prompt message to be used in a generative AI model is, "We have recognized from the user's facial expression that they want to relax, so please suggest a hairstyle that emphasizes relaxation."

[0609] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0610] Step 1:

[0611] The terminal provides an interface for the user to take a picture of their face and select their desired hairstyle. Based on the data entered by the user, the face image and hairstyle information are combined into a data packet and sent to the server. The input is the user's face image and style information, and the output is a data packet.

[0612] Step 2:

[0613] The server analyzes the received data packets. It processes the facial image using a facial recognition API and extracts the user's facial characteristics as digital data. This process yields facial feature point data. The input is a facial image, and the output is facial feature point data.

[0614] Step 3:

[0615] Furthermore, the server uses an emotion analysis API to analyze the user's emotional state from their facial image. This analysis determines the user's current emotional state and outputs it as emotion data. The input is a facial image, and the output is emotion data.

[0616] Step 4:

[0617] Based on the aforementioned facial feature point data and emotion data, the server utilizes a generative AI model to generate hairstyle candidates suitable for the user. In this process, prompts are used to provide input to the AI ​​model requesting the generation of specific styles, and a list of hairstyle candidates is output.

[0618] Step 5:

[0619] The server prioritizes and sorts the generated hairstyle candidates according to the user's emotional state. Based on emotional feedback, it adjusts the order of the styles and outputs a reorganized list of candidates. The input is hairstyle candidates, and the output is a reorganized list of hairstyles.

[0620] Step 6:

[0621] A rearranged list of hairstyles is presented to the user through a visual device, allowing for selection. The user chooses their desired style, and this selection is recorded. The input is the rearranged list of hairstyles, and the output is the user's final selection.

[0622] Step 7:

[0623] The final selected style information is provided to beauty professionals for reference during treatment. This information also includes treatment considerations based on the user's emotional state. The input is the user's final selection, and the output is the treatment information provided to beauty professionals.

[0624] The specific processing unit 290 transmits the result of the specific processing to the headset terminal 314. In the headset terminal 314, the control unit 46A causes the speaker 240 and display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0625] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet Search).<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0626] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and specific processing may also be performed by the headset terminal 314.

[0627] [Fourth Embodiment]

[0628] Figure 7 shows an example of the configuration of the data processing system 410 according to the fourth embodiment.

[0629] As shown in Figure 7, the data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[0630] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0631] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a controlled object 443. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and controlled object 443 are also connected to the bus 52.

[0632] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0633] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0634] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0635] The controlled object 443 includes a display device, LEDs in the eyes, and motors that drive the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the robot 414's emotions can be expressed by controlling these motors. Furthermore, the robot 414's facial expressions can also be expressed by controlling the illumination state of the LEDs in its eyes.

[0636] Figure 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Figure 8, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0637] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0638] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0639] In robot 414, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0640] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0641] This invention relates to a method and apparatus for implementing an "AI haircut ordering system." The system generates an optimal hairstyle based on the user's facial image and desired hairstyle information, and presents it to the user in advance, thereby reducing the gap between the user's wishes and the actual finished look.

[0642] User actions

[0643] First, the user downloads and launches a dedicated application on their device. Using the application, the user takes a photo of their face and selects their desired hairstyle. This information is then used as input for the system.

[0644] Terminal processing

[0645] The device generates a data packet to send a facial image and desired hairstyle information to the server. The data packet includes the user's individual facial image and information about the style selected by the user. If the user's allergy information has been registered in advance, that information is also included.

[0646] Server Processing

[0647] The server analyzes the received data and uses an algorithm to analyze facial image features to extract the user's facial contours and hair characteristics. Next, a generative AI model is used to generate multiple hairstyle candidates based on the analyzed features. This generative AI model has been trained on past hairstyle datasets and the latest trend information, and provides the optimal candidate based on this.

[0648] The multiple style options generated from the analysis results are sent back to the terminal for user confirmation. The user can select the style they like best from the presented options. This selected style is recorded as the final order and provided to the beauty professional.

[0649] Specific example

[0650] For example, if a user wants a "short bob style," they log into the system, upload a photo of their face, search for "short bob," and select their favorite style. The system analyzes the facial image, generates several suitable bob style variations, and presents them to the user. The style B selected by the user is ultimately recorded and provided to a beauty professional along with related information.

[0651] This system allows users to see an image of the finished look beforehand, minimizing the gap between their expectations and the final result, and making their salon experience more satisfying.

[0652] The following describes the processing flow.

[0653] Step 1:

[0654] The user launches a smartphone application, follows the instructions to take a picture of their face, and searches for and selects their preferred hairstyle. The image and selected style information are recorded on the device.

[0655] Step 2:

[0656] The device generates facial images and hairstyle information as data packets and sends them to the server. The data may also include the user's allergy information.

[0657] Step 3:

[0658] The server analyzes the received data, examining the user's facial contours and hair texture from the facial image. A facial recognition algorithm is used to extract detailed hair characteristics.

[0659] Step 4:

[0660] The server runs an AI model and generates multiple hairstyle options that suit the user's characteristics based on the analysis results. This model takes trend information into account and suggests the optimal style.

[0661] Step 5:

[0662] The server sends the generated hairstyle options to the terminal.

[0663] Step 6:

[0664] The device presents the user with several style options it has received. The user reviews them and selects the style they like best.

[0665] Step 7:

[0666] The terminal records the style information selected by the user as the final order and sends it to a server for communication with beauty professionals.

[0667] Step 8:

[0668] The server provides beauty professionals with details on the selected style, the treatment method, and the necessary products. This ensures that the treatment at the salon runs smoothly.

[0669] (Example 1)

[0670] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0671] It is essential to reduce the gap between the hairstyle the user desires and the actual finished look, and to provide the optimal style that suits each individual's characteristics. Furthermore, it is necessary to perform safe and effective treatments based on the user's health information.

[0672] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0673] In this invention, the server includes means for analyzing facial images received from the user, desired hairstyle information, and registered health information; means for presenting multiple hairstyle candidates generated using the analysis results and a generation AI model; and means for providing the hairstyle information selected by the user to a specialist to support the provision of optimal service. This makes it possible to suggest a style suitable for the user and to provide safe and highly satisfying beauty services.

[0674] A "user" refers to an individual who uses the system to receive suggestions and make selections for their own hairstyle.

[0675] A "face image" is digital image data that includes the features of the user's face.

[0676] "Hair style information" refers to digital data and attribute information related to the hairstyle desired by the user.

[0677] "Health information" refers to information related to the user's health status and allergies.

[0678] "Analysis method" refers to a method of processing facial images and hairstyle information based on received data and extracting their characteristics.

[0679] A "generative AI model" is an artificial intelligence program that creates optimal hairstyle options for a user based on past data and trends.

[0680] "Presentation method" refers to the method of displaying the analyzed and generated hairstyle candidates to the user.

[0681] A "specialist" refers to a professional who possesses the skills and knowledge related to beauty and provides treatments and advice to users.

[0682] This section describes the embodiments for carrying out the invention. This system, an "AI haircut ordering system," utilizes smart devices and computer servers to facilitate end-user use. The overall system overview and operation are described in detail below.

[0683] Users access the platform by downloading a dedicated application to their smart device and launching it. Within the application, users take a photo of their face using the camera and select their desired hairstyle. This automatically sends the user's face image and hairstyle information to the system.

[0684] The device generates a data packet containing the captured facial image and the user's selected hairstyle information, and prepares it for transmission to the server. This data packet may also include pre-registered health information.

[0685] The server analyzes the received data packets. On the server, it extracts facial features using image analysis software (e.g., OpenCV or DeepFace). Based on the analyzed data, it generates various hairstyle candidates using a generative AI model (e.g., an AI program built with TensorFlow). This AI model has been trained on historical data and the latest trends, and can suggest the optimal style based on the user's preferences and characteristics.

[0686] The generated hairstyle options are sent to the user's device, allowing the user to select the style that best suits their preferences from among several options. This selected information is shared with beauty professionals and related staff and used to provide the user with the most suitable beauty services.

[0687] As a concrete example, after a user logs into the app, they upload a photo of themselves using the "camera" function, and then search for and select a "short bob" style. Based on this information, the server inputs a prompt message to the AI ​​model saying, "Generate a short bob style that suits these facial features," and generates an appropriate style.

[0688] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0689] Step 1:

[0690] The user launches a dedicated application on their smart device, takes a picture of their face with the camera, and selects their desired hairstyle. The input includes the facial image data the user has captured and the selected hairstyle information. This information is saved on the device based on the user's actions.

[0691] Step 2:

[0692] The device generates a data packet containing saved facial images and hairstyle information, as well as health information previously registered by the user. This data packet is structured in an appropriate format (e.g., JSON format) and prepared for transmission to the server.

[0693] Step 3:

[0694] The server analyzes data packets received from the terminal. The input is data packets, and the facial images contained within them are processed using image analysis software (e.g., OpenCV) to extract the user's facial features. The output is a feature vector containing facial contours and hair texture information.

[0695] Step 4:

[0696] The server uses a generative AI model to generate hairstyle candidates based on extracted facial features and input hairstyle information. The input consists of feature vectors, hairstyle information, and a prompt message, such as "Generate hairstyles that match these facial features." The output includes image data of multiple hairstyle candidates.

[0697] Step 5:

[0698] The image data of hairstyle candidates generated on the server is sent back to the terminal for user confirmation. The terminal displays the candidate styles on the screen for the user. Based on this information, the user can select the most suitable hairstyle.

[0699] Step 6:

[0700] The hairstyle information selected by the user is recorded as the final order and provided to the stylist. The server transmits this information to the stylist in the appropriate format to help prepare the beauty service for the user.

[0701] (Application Example 1)

[0702] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0703] Traditionally, hairstyle simulations and selections have often relied on static images, leading to discrepancies between the simulated look and the actual finished style. This makes it difficult for users to achieve satisfactory results in salons, resulting in dissatisfaction. Furthermore, there is a lack of sufficient means for users to visually try out suitable styles in real time.

[0704] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0705] In this invention, the server includes means for analyzing facial information received from the user and the selected appearance information; means for presenting a plurality of appearance candidates generated based on the analysis results; means for providing the selected appearance information; and means for displaying the change in appearance using the user's head image. This allows the user to visually try out different styles in real time using a smart device and select the appearance that best matches their preferences.

[0706] "Facial information received from the user" refers to image data of the user's face, which is used to analyze facial features.

[0707] "Selected appearance information" refers to information about the appearance and style desired by the user, and is data used to select appearances.

[0708] "Means of analysis" refers to the processes and technologies used to determine the suitability of a user's characteristics and style based on the received data.

[0709] "Means for presenting multiple candidate appearances" refers to a method that has the function of visually showing the user multiple styles generated based on the analysis results.

[0710] "Means of providing selected appearance information" refers to a system that stores data on the appearance selected by the user and provides it to external assistance as needed.

[0711] A "device that displays changes in appearance using a user's head image" is a device that projects changes in appearance onto a real image of the user's head, providing a visual simulation.

[0712] This invention provides an interactive appearance simulation system. The server performs an appearance simulation using a face image received from the user's smart device and selected appearance information. The server analyzes the face image using image processing software such as OpenCV and TensorFlow for executing generative AI models, and extracts individual user features. This makes it possible to identify characteristics such as the shape and dimensions of the user's face and hair texture.

[0713] Based on the identified features, the server uses a pre-trained AI model to generate multiple appearance candidates that match those features. These candidates reflect the latest trends, providing users with a variety of choices.

[0714] The device displays generated appearance options in real time on the smart glasses' display. Users can visually try out various appearances through the glasses and easily switch between them using voice and eye-tracking sensors.

[0715] For example, if a user wants to try a "casual look," they can use a prompt to instruct the AI ​​model to "generate multiple variations of casual styles from the user's facial image and display them in real time." Following this prompt, the AI ​​will present the user with the most suitable look options, which the user can then review and select their preferred look. This system allows users to easily try out different looks in a physical store and choose the one they like best, resulting in a highly satisfying experience.

[0716] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0717] Step 1:

[0718] The user takes a facial image using a smart device and selects desired appearance information. This information is entered into the device. The device then generates a data packet containing the facial image and selected appearance information and sends it to the server.

[0719] Step 2:

[0720] The server extracts the user's face image and appearance information from the received data packets. The server uses OpenCV to analyze the face image and identify facial features. Specifically, it detects the shape, dimensions, and hair texture of the face. The output of this process is a numerical representation of the facial features.

[0721] Step 3:

[0722] The server inputs the obtained facial features into a generating AI model and generates appearance candidates based on the prompt "Generate multiple appearance variations from the user's face image." The AI ​​model considers pre-trained trend information and outputs the most suitable appearance.

[0723] Step 4:

[0724] The generated appearance options are sent to the terminal by the server. The terminal displays the appearance changes in real time on the smart glasses' display. The user can visually confirm the appearance through the glasses and switch between options using gaze or voice commands.

[0725] Step 5:

[0726] The user's final selected appearance information is recorded on the device and uploaded to the server. This recorded information is then provided to the relevant service providers and used for future service improvements and personalized support.

[0727] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0728] This invention relates to an "AI haircut ordering system" that combines a "emotion engine" that acquires a user's facial image and desired hairstyle information and recognizes the user's emotions. The system generates hairstyle candidates that take into account the user's hair characteristics and emotions, and provides them to beauty professionals, thereby providing a service that fits the user's wishes and emotions.

[0729] User actions

[0730] The user downloads and launches a smartphone application. After launching, they use their device to take a photo of their face and select their desired hairstyle. The application uses this information as system input.

[0731] Terminal processing

[0732] The device forms a data packet containing the captured facial image and the user's selected hairstyle information, and sends it to the server. If the user's allergy information has been registered in advance, that information will also be included in the data.

[0733] Processing by the emotion engine

[0734] The server uses an emotion engine to analyze received facial images, determining the user's emotional state from their facial expressions. This emotion recognition helps evaluate whether the style chosen by the user fits their emotional state.

[0735] Server Processing

[0736] Taking into account the user's emotions recognized by the emotion engine, the server uses a generative AI model to generate multiple hairstyle options that reflect the user's characteristics. The order in which the generated styles are presented may be adjusted based on the emotions. Information suggesting precautions for beauty professionals during treatment can also be generated.

[0737] Presenting the style

[0738] The terminal presents the user with multiple style options received from the server. The user selects their favorite style from the presented options, which is then recorded as the final order and provided to the beauty professional.

[0739] Specific example

[0740] For example, if a user requests "long curls," but the emotion engine recognizes from the user's current facial expression that they want to "relax," the system will prioritize suggesting styles that emphasize "relaxation." The style chosen by the user is recorded as the final order, and during the treatment, the beauty professional is provided with notes based on the emotion recognition.

[0741] This system allows us to provide hairstyle services that suit both the user's preferences and emotional state, thereby improving the user experience.

[0742] The following describes the processing flow.

[0743] Step 1:

[0744] The user launches a smartphone application, follows the instructions to take a picture of their face, and selects their desired hairstyle. The face image and selected style information are then entered into the device.

[0745] Step 2:

[0746] The device forms a data packet containing the captured facial image and selected hairstyle information, and sends it to the server. The data may also include the user's allergy information.

[0747] Step 3:

[0748] The server analyzes the received facial image using an emotion engine, recognizing the user's emotional state from their facial expressions. This allows for responses tailored to the user's emotions, such as a style that makes them feel safe and relaxed, or a style that energizes them.

[0749] Step 4:

[0750] The server analyzes hair characteristics based on the facial image and selected hairstyle information, and uses a generative AI model to generate multiple hairstyle candidates suitable for the user. The priority of the style candidates is adjusted based on the recognized emotional state.

[0751] Step 5:

[0752] The server sends the generated hairstyle suggestions and their detailed information to the terminal. This includes style images, key cutting points, recommended products, and emotionally-driven suggestions.

[0753] Step 6:

[0754] The device presents the user with multiple style options it has received. The user selects the style that seems to fit best from the presented options. This selection is recorded as the final order.

[0755] Step 7:

[0756] The selected style information is sent again from the terminal to the server and provided to the beauty professional as the final decision.

[0757] Step 8:

[0758] The server provides beauty professionals with detailed information, including the selected style and considerations based on the user's emotional state to be taken into account when performing the treatment. This ensures that the treatment aligns with the user's emotions, leading to increased satisfaction.

[0759] (Example 2)

[0760] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0761] Traditionally, there have been no systems that consider the user's emotional state when selecting a hairstyle, resulting in users not achieving their desired level of satisfaction. Furthermore, in order to suggest an appropriate style, it is necessary to accurately reflect information about each individual's head characteristics and health, but previous systems have not adequately addressed this issue.

[0762] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0763] In this invention, the server includes means for analyzing a facial image received from the user and selected style information, means for recognizing the user's emotional state from the facial image using an emotion engine, and means for a generating AI model to generate and present multiple style candidates based on the analysis results and emotional state. This makes it possible to suggest styles suitable for individual characteristics while taking into account the user's emotional state, and further select appropriate products and services based on health information.

[0764] A "user" refers to an individual who uses the system to provide their facial image and style information in order to receive beauty services.

[0765] "Facial images" refer to image data captured in order to analyze a user's facial expressions and head characteristics.

[0766] "Style information" refers to information about the hairstyle desired by the user, including the type and characteristics of the selected style.

[0767] An "emotion engine" refers to software or algorithms that analyze a user's facial image and recognize their emotional state from their facial expressions.

[0768] A "generative AI model" refers to artificial intelligence technology that generates optimal style candidates based on the user's characteristics and emotional state.

[0769] "Hairstyle options" refers to the multiple hairstyle choices suggested to the user, from which the user selects the one that best suits them.

[0770] "Health-related information" refers to allergy information and other health status data provided by users, which is used to select appropriate products and services.

[0771] A "beauty professional" refers to a specialized service provider who performs hair styling treatments for users.

[0772] The specific operation of the system in an embodiment for carrying out this invention is shown below.

[0773] First, the user launches a dedicated application installed on their mobile device. Using the application, the user can take a photo of their face and input their desired style information. This face photo and style information are stored on the device as data packets. If necessary, the user can register health information in the application and record allergies and special precautions.

[0774] The terminal sends the generated data packets to the server. This transmission takes place via the internet, ensuring high speed and security. The server is equipped with advanced image analysis software and an emotion engine, which evaluates the user's emotional state from the received facial images. This emotion engine is based on algorithms that analyze the user's facial features and identify emotions in real time.

[0775] Furthermore, the server utilizes a generative AI model to generate multiple style options that suit the user's characteristics and emotions. These generated style options are then presented in a prioritized manner based on the user's emotional state. For example, if the server determines that the user desires "long curls" but also wants to relax, it will prioritize suggesting styles that evoke a strong sense of relaxation.

[0776] Style information transmitted from the server is displayed on the terminal. The user can view style options on the screen and select their preferred one. The selected style is recorded as the final choice and, when provided to a beauty professional, is accompanied by treatment advice based on the emotional criteria of the suggested style.

[0777] Throughout this entire process, users can receive personalized beauty services tailored to their emotional state and preferences.

[0778] An example of a prompt message might be, "Suggest a hairstyle suitable for a user who wants to relax." This allows the system to generate and provide a style that incorporates the user's emotions and desires.

[0779] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0780] Step 1:

[0781] The user launches an application installed on their mobile device and takes a photo of their face. The application provides an input screen where the user can select their desired hairstyle. This input process yields both the facial image data and the selected style information.

[0782] Step 2:

[0783] The terminal composes the input facial image data and style information into a data packet. This data packet also includes health information registered by the user, if necessary. The terminal prepares to transmit this data packet to the server via the network connection.

[0784] Step 3:

[0785] The server receives data packets over the network. Based on the received data, it processes the facial image using image analysis software to extract the user's facial features. This outputs facial feature quantities from the image data for input into the emotion engine.

[0786] Step 4:

[0787] The server inputs the extracted facial features into the emotion engine. The emotion engine uses these features to recognize the user's emotional state. This process allows for the recognition of emotional states such as "joy" and "relaxation."

[0788] Step 5:

[0789] The server inputs emotional state and style information into a generative AI model, which generates style candidates suitable for the user's characteristics and emotional state. The generative AI model uses machine learning techniques to output multiple style candidates from this data.

[0790] Step 6:

[0791] The server prioritizes the generated style candidates based on the user's emotional state. For example, if the user is perceived as wanting to relax, styles that emphasize "relaxation" will be prioritized. This prioritized style information is then output.

[0792] Step 7:

[0793] The terminal presents the user with style options received from the server. The user can view multiple style options on the screen and select the one they like best. This selection then provides the final selected style information.

[0794] Step 8:

[0795] The terminal sends the user's selected style information to the server as the final order. The server provides the beauty professional with treatment considerations based on the selected style and emotions, helping them to provide a service that matches the user's wishes and emotions.

[0796] (Application Example 2)

[0797] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0798] Conventional hairstyle suggestion systems have limitations in improving the user experience because they make suggestions without considering the user's emotional state. Furthermore, it was difficult to suggest the most suitable hairstyle to the customer in real time. These challenges made it difficult to provide services and customer care that would satisfy users.

[0799] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0800] In this invention, the server includes means for analyzing a facial image received from the user and information on the desired hairstyle, means for analyzing the user's emotional state in real time using a visual device, and means for adjusting suggested hairstyles based on the user's emotional state. This makes it possible to suggest hairstyles that are suitable for the user's emotions.

[0801] "Means for analyzing facial images and desired hairstyle information received from a user" refers to a processing device that uses digital technology to analyze facial image data and information about desired hairstyles obtained from a user, and extracts style candidates based on the results.

[0802] "Means for presenting multiple hairstyle options generated based on analysis results" refers to an interface for visually or electronically suggesting multiple hairstyles generated from the analysis results to the user.

[0803] "Means for providing selected hairstyle information to beauty professionals" refers to an information provision device that transmits information about the hairstyle selected by the user to beauty professionals, enabling them to perform treatments based on that information.

[0804] "Means for analyzing a user's emotional state in real time using visual devices" refers to a technology that uses visual devices such as cameras to instantly determine emotions from a user's facial expressions and other visual information, and then analyzes that information.

[0805] "Means for adjusting suggested hairstyles based on the user's emotional state" refers to an algorithm or system that adjusts the priority and content of suggested hairstyles according to the results of emotion analysis, thereby providing the user with the most suitable option based on their mood and emotions.

[0806] In implementing this invention, the user first accesses the system using a terminal device such as a smartphone or tablet. The user takes a picture of their face and selects their desired hairstyle. This information is digitized on the terminal and transmitted to the server. The server utilizes image recognition technology and sentiment analysis software to analyze the received face image and hairstyle information.

[0807] Specifically, the server analyzes the user's face using a facial recognition API (e.g., AWS Rekognition) and evaluates the user's emotional state in real time using an emotion analysis API (e.g., Microsoft Azure Emotion API). Based on this analysis data, it uses a generative AI model (e.g., OpenAI GPT-3) to generate multiple hairstyle candidates that fit the user's emotions.

[0808] The generated hairstyles are prioritized and rearranged based on the user's emotional state and hair characteristics. For example, if the user indicates that they want to relax, the server will prioritize presenting styles that are appropriate for that emotion.

[0809] The presented style options are shown to beauty professionals via visual devices such as smart glasses, allowing them to perform treatments that take the user's emotional state into consideration. Smooth interaction between the user and the beauty professional leads to the provision of highly satisfying services.

[0810] For example, if a user appears tense upon arriving at the salon, the system might prioritize styles that promote relaxation. In this way, the system provides a hairstyle that best matches the user's preferences and emotional state.

[0811] An example of a prompt message to be used in a generative AI model is, "We have recognized from the user's facial expression that they want to relax, so please suggest a hairstyle that emphasizes relaxation."

[0812] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0813] Step 1:

[0814] The terminal provides an interface for the user to take a picture of their face and select their desired hairstyle. Based on the data entered by the user, the face image and hairstyle information are combined into a data packet and sent to the server. The input is the user's face image and style information, and the output is a data packet.

[0815] Step 2:

[0816] The server analyzes the received data packets. It processes the facial image using a facial recognition API and extracts the user's facial characteristics as digital data. This process yields facial feature point data. The input is a facial image, and the output is facial feature point data.

[0817] Step 3:

[0818] Furthermore, the server uses an emotion analysis API to analyze the user's emotional state from their facial image. This analysis determines the user's current emotional state and outputs it as emotion data. The input is a facial image, and the output is emotion data.

[0819] Step 4:

[0820] Based on the aforementioned facial feature point data and emotion data, the server utilizes a generative AI model to generate hairstyle candidates suitable for the user. In this process, prompts are used to provide input to the AI ​​model requesting the generation of specific styles, and a list of hairstyle candidates is output.

[0821] Step 5:

[0822] The server prioritizes and sorts the generated hairstyle candidates according to the user's emotional state. Based on emotional feedback, it adjusts the order of the styles and outputs a reorganized list of candidates. The input is hairstyle candidates, and the output is a reorganized list of hairstyles.

[0823] Step 6:

[0824] A rearranged list of hairstyles is presented to the user through a visual device, allowing for selection. The user chooses their desired style, and this selection is recorded. The input is the rearranged list of hairstyles, and the output is the user's final selection.

[0825] Step 7:

[0826] The final selected style information is provided to beauty professionals for reference during treatment. This information also includes treatment considerations based on the user's emotional state. The input is the user's final selection, and the output is the treatment information provided to beauty professionals.

[0827] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the controlled object 443 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0828] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet Search).<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0829] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the robot 414.

[0830] Furthermore, the emotion identification model 59, acting as an emotion engine, may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to a specific mapping, which is an emotion map (see Figure 9). Similarly, the emotion identification model 59 may also determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[0831] Figure 9 shows an emotion map 400 in which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. The closer to the center of the concentric circles, the more primitive the emotions are located. Further out of the concentric circles, emotions representing states and actions arising from mental states are located. Emotion is a concept that includes feelings and mental states. On the left side of the concentric circles, emotions that are generally generated from reactions occurring in the brain are located. On the right side of the concentric circles, emotions that are generally induced by situational judgment are located. Above and below the concentric circles, emotions that are generally generated from reactions occurring in the brain and induced by situational judgment are located. In addition, the emotion of "pleasure" is located on the upper side of the concentric circles, and the emotion of "displeasure" is located on the lower side. Thus, in the emotion map 400, multiple emotions are mapped based on the structure in which emotions arise, and emotions that are likely to occur simultaneously are mapped close together.

[0832] These emotions are distributed at the 3 o'clock position on the Emotion Map 400, and usually fluctuate between feelings of security and anxiety. In the right half of the Emotion Map 400, situational awareness takes precedence over internal feelings, resulting in a calm impression.

[0833] The inside of the Emotion Map 400 represents inner thoughts, while the outside represents actions. Therefore, the further you go from the outside of the Emotion Map 400, the more visible (expressed in actions) your emotions become.

[0834] Here, human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. Similarly, in robots, cars, motorcycles, etc., emotions can be created based on various balances, such as posture and battery level. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. The emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on a system for analyzing brain physiological signals of speech emotion recognition and emotion, Tokushima University, doctoral dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map contains emotions belonging to a region called "response," where sensation is dominant. The right half of the emotion map contains emotions belonging to a region called "situation," where situational awareness is dominant.

[0835] The emotion map defines two emotions that promote learning. One is the emotion around the middle of the negative "repentance" and "reflection" on the situation side. In other words, it is when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is the emotion around the positive "desire" on the reaction side. In other words, it is when the robot has positive feelings such as "I want more" or "I want to know more."

[0836] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​representing each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple training data sets, which are combinations of user input and emotion values ​​representing each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions located close together have similar values, as shown in the emotion map 900 in Figure 10. Figure 10 shows an example where multiple emotions such as "reassured," "calm," and "confident" have similar emotion values.

[0837] The above description primarily focuses on the functions of the data processing device 12 in relation to this disclosure. However, the system related to this disclosure is not necessarily implemented on a server. The system related to this disclosure may be implemented as a general information processing system. This disclosure may be implemented, for example, as a software program that runs on a personal computer or as an application that runs on a smartphone. The method related to this disclosure may be provided to users in SaaS (Software as a Service) format.

[0838] In the above embodiment, an example was given in which a specific process is performed by a single computer 22. However, the technology of this disclosure is not limited thereto, and a distributed processing of the specific process may be performed by multiple computers, including computer 22. For example, a data generation model 58 may be provided in an external device of the data processing device 12, and the external device may generate data according to the input data.

[0839] In the above embodiment, an example was given in which the specific processing program 56 is stored in the storage 32, but the technology of this disclosure is not limited thereto. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-temporary storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-temporary storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes specific processing according to the specific processing program 56.

[0840] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[0841] Furthermore, it is not necessary to store the entirety of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store the entirety of the specific processing program 56 in the storage 32; it is acceptable to store only a portion of the specific processing program 56.

[0842] The following types of processors can be used as hardware resources to perform specific processing. Examples of processors include a CPU, a general-purpose processor that functions as a hardware resource to perform specific processing by executing software, i.e., a program. Other examples of processors include dedicated electrical circuits, such as FPGAs (Field-Programmable Gate Arrays), PLDs (Programmable Logic Devices), or ASICs (Application Specific Integrated Circuits), which have circuit configurations specifically designed to perform specific processing. All of these processors have built-in or connected memory, and all of them perform specific processing by using memory.

[0843] The hardware resource that performs a specific process may consist of one of these various processors, or it may consist of a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Alternatively, the hardware resource that performs a specific process may consist of a single processor.

[0844] Examples of configurations using a single processor include, firstly, a configuration in which one or more CPUs and software are combined to form a single processor, and this processor functions as a hardware resource that performs a specific process. Secondly, there is a configuration using a processor that realizes the functions of the entire system, including multiple hardware resources that perform a specific process, on a single IC chip, as exemplified by SoCs (System-on-a-chip). In this way, a specific process is realized using one or more of the above types of processors as hardware resources.

[0845] Furthermore, the hardware structure of these various processors can more specifically utilize electrical circuits that combine circuit elements such as semiconductor devices. Also, the specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps can be deleted, new steps added, or the processing order rearranged, as long as it does not deviate from the main purpose.

[0846] The descriptions and illustrations presented above are detailed explanations of the technical aspects of this disclosure and are merely examples of the technical aspects. For example, the above descriptions of the structure, function, operation, and effect are examples of the structure, function, operation, and effect of the technical aspects of this disclosure. Therefore, it goes without saying that you may delete unnecessary parts, add new elements, or replace elements in the descriptions and illustrations presented above, as long as you do not deviate from the essence of the technical aspects of this disclosure. Furthermore, in order to avoid confusion and facilitate understanding of the technical aspects of this disclosure, explanations of common technical knowledge and the like that do not require special explanation to enable the implementation of the technical aspects of this disclosure have been omitted from the descriptions and illustrations presented above.

[0847] All documents, patent applications, and technical standards described herein are incorporated by reference to the same extent as if each individual document, patent application, and technical standard were specifically and individually noted as being incorporated by reference.

[0848] The following is further disclosed regarding the embodiments described above.

[0849] (Claim 1)

[0850] A means for analyzing facial images received from users and information on desired hairstyles,

[0851] A means for presenting multiple hairstyle candidates generated based on the analysis results,

[0852] A means of providing selected hairstyle information to beauty professionals,

[0853] A system that includes this.

[0854] (Claim 2)

[0855] The system according to claim 1, which identifies the individual hair characteristics of a user by analyzing a facial image and generates an optimal style that suits those characteristics.

[0856] (Claim 3)

[0857] The system according to claim 1, comprising means for registering a user's allergy information and selecting an appropriate drug based on the registered information.

[0858] "Example 1"

[0859] (Claim 1)

[0860] A means for analyzing facial images received from users, desired hairstyle information, and registered health information,

[0861] A means for presenting multiple hairstyle candidates generated using the analysis results and the generation AI model,

[0862] A means of providing users with information on their chosen hairstyle to experts to support them in providing the best possible service,

[0863] A system that includes this.

[0864] (Claim 2)

[0865] The system according to claim 1, which identifies the individual head characteristics of a user by analyzing a facial image and generates an optimal hairstyle that suits those characteristics.

[0866] (Claim 3)

[0867] The system according to claim 1, comprising means for registering a user's health information and selecting an appropriate product based on the registered information.

[0868] "Application Example 1"

[0869] (Claim 1)

[0870] A means for analyzing facial information received from the user and selected appearance information,

[0871] A means for presenting multiple candidate appearances generated based on the analysis results,

[0872] Means for providing selected appearance information,

[0873] A device means for displaying changes in appearance using a user's head image,

[0874] A system that includes this.

[0875] (Claim 2)

[0876] The system according to claim 1, which identifies individual user characteristics by analyzing facial information and generates an appearance that matches those characteristics.

[0877] (Claim 3)

[0878] The system according to claim 1, comprising means for registering user attribute information and selecting appropriate action based on the registered information.

[0879] "Example 2 of combining an emotion engine"

[0880] (Claim 1)

[0881] A means for analyzing the facial image received from the user and the selected style information,

[0882] A means of recognizing a user's emotional state from a facial image using an emotion engine,

[0883] A means by which a generative AI model generates and presents multiple style candidates based on analysis results and emotional state,

[0884] A means of adjusting the order in which the presented style options are displayed based on the user's emotions,

[0885] A means of providing beauty professionals with selected style information and treatment precautions based on emotions,

[0886] A system that includes this.

[0887] (Claim 2)

[0888] The system according to claim 1, which provides means for generating an optimal style adapted to the individual head characteristics and emotions of a user through facial image analysis and emotion recognition.

[0889] (Claim 3)

[0890] The system according to claim 1, which includes means for registering user health information and selecting appropriate products or services based on this information.

[0891] "Application example 2 of combining emotional engines"

[0892] (Claim 1)

[0893] A means for analyzing facial images received from users and information on desired hairstyles,

[0894] A means for presenting multiple hairstyle candidates generated based on the analysis results,

[0895] A means of providing selected hairstyle information to beauty professionals,

[0896] A means of analyzing a user's emotional state in real time using a visual device,

[0897] A means of adjusting suggested candidates based on the user's emotional state,

[0898] A system that includes this.

[0899] (Claim 2)

[0900] The system according to claim 1, which identifies the individual hair characteristics of a user by analyzing a facial image and generates an optimal style that suits those characteristics.

[0901] (Claim 3)

[0902] The system according to claim 1, comprising means for registering a user's allergy information and selecting an appropriate drug based on the registered information. [Explanation of Symbols]

[0903] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Devices 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robots< / url:> < / url:> < / url:> < / url:>

Claims

1. A means for analyzing facial images received from users and information on desired hairstyles, A means for presenting multiple hairstyle candidates generated based on the analysis results, A means of providing selected hairstyle information to beauty professionals, A system that includes this.

2. The system according to claim 1, further comprising means for identifying the individual hair characteristics of a user by analyzing a facial image and generating an optimal style that suits those characteristics.

3. The system according to claim 1, further comprising means for registering the user's allergy information and selecting an appropriate drug based on the registered information.

Images