System

Abstract

A system is provided.SOLUTION: A system comprising: an image input means for allowing a user to simulate a hairstyle or a hair color for his / her own image; an image processing means for generating an image of the simulated hairstyle or hair color; a display means for allowing the user to confirm the generated image; an information transmission means for sharing the generated image with a beauty salon when reserving the beauty salon; and a display means for allowing the beauty salon to display and confirm the received image.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] In today's world, users rely on verbal instructions at hair salons to achieve the hairstyle and hair color they envision. However, it is extremely difficult to accurately convey a specific image of color or style through words alone, and as a result, the finished product often differs from what was envisioned. This problem is particularly pronounced in the field of hair color, where misunderstandings between the user and the stylist are likely to occur. Furthermore, if users are unable to communicate their desired image to the salon in advance, treatment time and chemical mixtures can be wasted. Furthermore, from the perspective of protecting user privacy, a secure and highly anonymous system is required. [Means for solving the problem]

[0005] To solve the above-mentioned problems, the present invention provides the following means. First, it has an image input means that allows a user to simulate hairstyles and hair colors using an image of themselves. Next, it has an image processing means that generates an image of the simulated hairstyle or hair color, and a display means that allows the user to check the generated image. Furthermore, it provides an information transmission means that shares the generated image with a hair salon and transmits it to the hair salon along with reservation information. The hair salon has a display means that displays the received image so that the stylist can check it. In addition, it has an image processing means that processes the user's facial photo to be anonymized, and also includes a function to send the anonymized image to the hair salon. It also provides a video generation means that allows the user to check the simulated hairstyle or hair color image in a video, and a function to simulate color changes due to seasons and weather. This allows the user and the hair salon to share the image more accurately, making it possible to achieve a satisfactory hair styling.

[0006] The "user image input means" is a means for a user to start simulating hairstyles and hair colors by uploading or photographing an image of themselves.

[0007] The "image processing means" refers to a means for applying hairstyle and hair color simulation to an uploaded or photographed image of a user to generate a visual image.

[0008] The "display means" is a means for visually presenting to the user an image of the hairstyle and hair color generated by the simulation.

[0009] The "information transmission means" is a means for transmitting the generated image and reservation information to the beauty salon.

[0010] The "hair salon display means" is a means for the hair salon to check the received image and use it as a reference for treatment.

[0011] "Image processing means" refers to a means of anonymizing a user's facial photograph to protect their privacy.

[0012] The "video generating means" is a means for converting the simulation results of hairstyles and hair colors, which are still images, into video format so that the user can check them in a moving state.

[0013] The "color change simulation function depending on the season and weather" is a function that allows the user to simulate how the selected hair color will change depending on the season and weather, and check it. [Brief explanation of the drawings]

[0014] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11]FIG. 3 is a sequence diagram showing a processing flow of the data processing system according to the first embodiment. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0015] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0016] First, the terms used in the following description will be explained.

[0017] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0018] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0019] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0020] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0021] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0022] [First embodiment]

[0023] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0024] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0025] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0026] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0027] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0028] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0029] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0030] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0031] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0032] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0033] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0034] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0035] The present invention relates to a system that allows users to simulate hairstyles and hair colors for their own images and share the results with beauty salons. In particular, the system provides a system that allows users to apply their preferred hairstyles and colors to their own facial photos and perform a simulation based on the results of the simulation, enabling more accurate treatments at beauty salons.

[0036] Overall system overview

[0037] This system is broadly composed of the following elements:

[0038] User Device

[0039] server

[0040] Beauty salon terminal

[0041] 1. User Device

[0042] The user terminal provides an interface for users to take or upload their own images and simulate hairstyles and hair colors. Specifically, the following functions are included:

[0043] Image input method: The user takes a photo of their face using the camera function or file upload function.

[0044] Display means: Simulation results are displayed to the user in still image or video format.

[0045] Information transmission means: The generated image and reservation information are sent to the beauty salon.

[0046] 2. Server

[0047] The server receives the image sent by the user and uses AI to apply hairstyles, colors, anonymize, etc. It also forwards the generated image to the hair salon.

[0048] Image processing method: Apply different hairstyles and colors to the user's face photo and perform simulations.

[0049] Image processing method: To protect privacy, the user's face photo is anonymized.

[0050] Video generation means: Converts the simulation results from still images into video format, allowing users to check the motion simulation.

[0051] Information transmission means: The generated image and reservation information are sent to the beauty salon.

[0052] 3. Beauty salon terminal

[0053] The salon terminal checks the image received from the server and projects it onto an AI mannequin, allowing the stylist to use it as a reference for the treatment.

[0054] Hair salon display means: The received image is displayed and used as a reference for treatment.

[0055] Specific examples

[0056] The program processing will be explained below using specific examples.

[0057] Example 1: Simulation flow

[0058] 1. User: Launches the app and takes or uploads a photo of their face.

[0059] The device sends the user's photo to the server.

[0060] 2. Server: The AI ​​module analyzes the received facial photo and cuts out the facial area.

[0061] 3. User: Select a hairstyle and color from the list provided within the app.

[0062] The terminal transmits the selection information to the server.

[0063] 4. Server: Apply the selected style and color to the user's image and generate the simulation result.

[0064] The generated image is sent back to the device.

[0065] 5. User: Check the simulation results using still images and videos, and make fine adjustments if necessary.

[0066] Example 2: Sharing information with a hair salon

[0067] 1. User: Makes a reservation at a beauty salon based on the simulation results.

[0068] The terminal transmits the reservation information and image data to the server.

[0069] 2. Server: Transfers the received reservation information and image data to the beauty salon terminal.

[0070] 3. Beauty salon terminal: The received image is projected onto the AI ​​mannequin and the stylist checks it.

[0071] 4. Hair salon: The stylist performs the treatment based on the image received.

[0072] The above is an embodiment of the present invention. The present invention facilitates communication between the user and the stylist, and allows for more accurate hair styling.

[0073] The processing flow will be explained below.

[0074] Step 1:

[0075] A user launches the app and creates a new account or logs in with an existing account.

[0076] Step 2:

[0077] The user uses the camera function within the app to take a photo of their face or upload an existing photo, and the device sends this image data to the server.

[0078] Step 3:

[0079] The server analyzes the facial photos it receives using an AI module, identifies and cuts out the facial features, and stores the image data in a database after facial recognition processing is complete.

[0080] Step 4:

[0081] The user selects their preferred hairstyle and color from a list of hairstyle and color templates provided within the app, and the device sends the selection information to the server.

[0082] Step 5:

[0083] The server passes the selected hairstyle and color information to the AI ​​module, which applies it to the user's image, and returns the resulting image to the device.

[0084] Step 6:

[0085] The device displays the generated images to the user, or if the user wants to simulate the action, they request that the images be generated in animated format.

[0086] Step 7:

[0087] The server receives the request and generates the simulation results in the form of a video, which is then sent back to the device and displayed for the user to review.

[0088] Step 8:

[0089] If the user is satisfied with the simulation results, they make a reservation at the beauty salon. They enter the necessary information into the reservation form, attach the simulation results data, and submit it. The terminal then sends the reservation information and image data to the server.

[0090] Step 9:

[0091] The server receives the reservation information and image data and transfers them to the appropriate salon terminal.

[0092] Step 10:

[0093] The image data received by the salon terminal is projected onto an AI mannequin and displayed for the stylist to check.

[0094] Step 11:

[0095] At the salon, the stylist plans the combination of medications and treatment time based on the user's reservation information and the simulation results, and then actually performs the treatment.

[0096] Step 12:

[0097] Users can check the results of the treatment and share feedback within the app if necessary.

[0098] Example 1

[0099] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0100] When users receive treatment at a beauty salon, it is difficult for them to accurately communicate their desired hairstyle and hair color. Furthermore, when users simulate hairstyles and hair colors using photos of their own faces, it is difficult to get an idea from still images alone, and privacy is not protected if the images are not anonymized. Since it is not possible to simulate changes in color due to the seasons or weather, the results may not be as expected.

[0101] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0102] In this invention, the server includes a means for projecting the received image onto the AI ​​mannequin, a face detection means for extracting the user's face, and a video generation means for viewing the simulation results in video format. This allows the user to accurately communicate their desired hairstyle and hair color, and viewing the simulation results in video makes it easier to understand the image, while also protecting privacy through anonymity. Furthermore, by simulating color changes due to seasons and weather, it is possible to achieve the expected results.

[0103] "Image input means" refers to the means used by a user to take or upload a photo of their face.

[0104] The "image processing means" is a means for applying a hairstyle and hair color to a facial photo sent by a user and performing a simulation.

[0105] The "display means" is a means for displaying the generated simulation results so that the user can check them.

[0106] The "information transmission means" is a means for transmitting the generated simulation image to the beauty salon.

[0107] "Means for projecting the received image onto the AI ​​mannequin" refers to the means by which the beauty salon projects the received image onto the AI ​​mannequin and visually confirms it.

[0108] The "face detection means" is a means for detecting and extracting the user's face, and mainly uses a deep learning model.

[0109] The "moving image generating means" is a means for generating simulation results in the form of not only still images but also moving images, allowing the user to check the operation simulation.

[0110] "Image processing means" refers to a means of anonymizing a user's facial photograph to protect their privacy.

[0111] The "data compression means" is a means for compressing the generated image data to improve communication efficiency.

[0112] The "function to simulate color changes due to seasons and weather" is a function for simulating changes in hair color due to environmental conditions.

[0113] This system allows users to simulate hairstyles and hair colors on their own images and share them with hair salons. This allows users to check their desired style in advance and ensure smooth and accurate treatment at the salon. The system consists of three main components: a user terminal, a server, and a salon terminal.

[0114] User Device

[0115] The user device provides an interface that allows users to take or upload their own images and simulate hairstyles and hair colors. Specifically, it has the following functions. First, as an image input means, the camera function or file upload function of a smartphone or tablet is used. The user can launch the app and take or upload a photo of their face. Next, as a display means, the simulation results are displayed to the user in still image or video format. This allows the user to visually check various hairstyles and color variations. Finally, as an information transmission means, the generated simulation image and reservation information can be sent to the hair salon.

[0116] server

[0117] The server receives images sent by users and uses AI to apply hairstyles and colors, and anonymize them. It also has the function of transmitting the generated images to the hair salon. Specifically, the functions are as follows: First, as an image processing method, an AI module (e.g., a deep learning model using TensorFlow or PyTorch) is used to analyze the user's facial photo and cut out the facial area. Next, as an image processing method, anonymization is performed on the user's facial photo to protect privacy. Finally, as a video generation method, the simulation results of still images are converted into video format so that the user can check the movement simulation. Finally, as an information transmission method, the generated images and reservation information are sent to the hair salon.

[0118] Beauty salon terminal

[0119] The salon terminal checks the image received from the server and projects it onto an AI mannequin, allowing the stylist to use it as a reference for the treatment. Specifically, it has the following functions. First, as a salon display means, it displays the received image using a display or projector, allowing the stylist to check the treatment details. Furthermore, by projecting the received image onto the AI ​​mannequin, the style can be visually checked in real time.

[0120] Specific examples

[0121] Example 1: Simulation flow

[0122] 1. User: Launches the app and takes or uploads a photo of their face.

[0123] The device sends the user's photo to the server.

[0124] 2. Server: The received face photo is analyzed using an AI module and facial features are extracted. For example, modules using OpenCV or deep learning are used.

[0125] 3. User: Select a hairstyle and color from the list provided within the app.

[0126] The terminal transmits the selection information to the server.

[0127] 4. Server: Apply the selected style and color to the user's image and generate the simulation result.

[0128] The generated image is sent back to the device.

[0129] 5. User: Check the simulation results using still images and videos, and make fine adjustments if necessary.

[0130] Example prompt for a generative AI model:

[0131] Input image: A face photo uploaded by the user

[0132] Hairstyle: Classic Bob

[0133] Hair Color: Blonde

[0134] Output: Generates and returns a simulated image with the specified hairstyle and hair color.

[0135] Example 2: Sharing information with a hair salon

[0136] 1. User: Makes a reservation at a beauty salon based on the simulation results.

[0137] The terminal transmits the reservation information and image data to the server.

[0138] 2. Server: Transfers the received reservation information and image data to the beauty salon terminal.

[0139] 3. Beauty salon terminal: The received image is projected onto the AI ​​mannequin and the stylist checks it.

[0140] 4. Hair salon: The stylist performs the treatment based on the image received.

[0141] This allows users to efficiently simulate their desired hairstyle and color, facilitating smooth communication with the hairdresser. By using this system, users can check their appearance in advance, resulting in a more satisfying experience at the hairdresser.

[0142] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0143] Step 1:

[0144] User: Launches the app and takes or uploads a photo of their face.

[0145] Specific actions include taking a photo of your face using the camera on your smartphone or tablet, or selecting and uploading an existing image file from a folder.

[0146] Input: User's face photo (image file)

[0147] Output: Facial photo data is saved on the user's device.

[0148] Step 2:

[0149] Terminal: The acquired facial photo is compressed and sent to a server via the Internet.

[0150] Specifically, the image file is encoded in JPEG format and sent to the server via an HTTP POST request.

[0151] Input: Facial photo data (image file)

[0152] Output: Compressed face photo data is sent to the server.

[0153] Step 3:

[0154] Server: The received image is analyzed using an AI module, and facial features are detected and extracted.

[0155] Specifically, it applies a facial recognition algorithm using deep learning libraries such as OpenCV and TensorFlow to identify and cut out the facial area.

[0156] Input: Compressed face photo data

[0157] Output: Image data of the cut-out face area

[0158] Step 4:

[0159] User: Selects a hairstyle and color from a list provided through the in-app interface.

[0160] Specifically, you click on the drop-down menus and icons displayed within the app to select the desired hairstyle and color.

[0161] Input: User's selection information (hair style and color)

[0162] Output: The selected hairstyle and color information is saved on the user's device.

[0163] Step 5:

[0164] Terminal: Sends the selection information to the server.

[0165] Specifically, the selected hairstyle and color information is encoded in JSON format and sent to the server via an HTTP POST request.

[0166] Input: User's selection information (hair style and color)

[0167] Output: The selection is sent to the server.

[0168] Step 6:

[0169] Server: Based on the selection information, apply the simulation to the user's face photo.

[0170] Specifically, it uses generative AI models such as TensorFlow and PyTorch to apply hairstyles and colors to a user's image, generating a realistic simulation.

[0171] Input: Image data of the cut-out face, selection information (hairstyle and color)

[0172] Output: Image data of the simulation results

[0173] Example prompt for a generative AI model:

[0174] Input image: A face photo uploaded by the user

[0175] Hairstyle: Classic Bob

[0176] Hair Color: Blonde

[0177] Output: Generates and returns a simulated image with the specified hairstyle and hair color.

[0178] Step 7:

[0179] Server: Generates simulation results in video format and sends them back to the user's device.

[0180] Specifically, the simulation results of still images are converted into video format (MP4, etc.) and sent to the user terminal as an HTTP response.

[0181] Input: Still image data of simulation results

[0182] Output: The simulation result video data is sent to the user's device.

[0183] Step 8:

[0184] User: Review the returned simulation results and make any necessary adjustments.

[0185] Specifically, you can change the hairstyle or color again within the app, start the simulation again, and confirm the final result.

[0186] Input: Simulation result video data

[0187] Output: Final simulation result data

[0188] Step 9:

[0189] User: Sends the final simulation results and reservation information to the hair salon.

[0190] Specifically, the reservation information (date and time, desired treatment) and a simulation image are sent to the server.

[0191] Input: Final simulation result data, reservation information

[0192] Output: The final simulation results and reservation information are sent to the server.

[0193] Step 10:

[0194] Server: Transfers the received reservation information and image data to the beauty salon terminal.

[0195] Specifically, the reservation information and the simulation image are sent to the beauty salon terminal.

[0196] Input: Final simulation results and reservation information

[0197] Output: Reservation information and image data are sent to the salon terminal.

[0198] Step 11:

[0199] Beauty salon terminal: The received image data is projected onto an AI mannequin for the stylist to check.

[0200] Specifically, the received image is displayed using a display or projector.

[0201] Input: Image data, reservation information

[0202] Output: Treatment is performed based on the projected image data.

[0203] The above are the specific processing steps of the system, which are a series of steps to simulate the hairstyle and hair color desired by the user and reflect them in the treatment at the beauty salon.

[0204] (Application example 1)

[0205] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0206] Conventional hair simulation systems have difficulty in sharing simulation results between users and hair salons, which can lead to misunderstandings during treatment. It is also difficult to check and fine-tune simulation results in real time, making it difficult for users to accurately communicate their desired style.

[0207] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0208] In this invention, the server includes means for receiving a facial photo taken or uploaded by a user terminal and analyzing it with an AI module, means for applying a selected hairstyle and hair color to the user's facial photo and returning the generated image, and an AI module for receiving the user's facial photo and generating a hairstyle and color simulation result in real time, allowing the user to simulate a hairstyle and hair color for their own image, check the result in real time, and share it with a hair salon.

[0209] "User terminal" means a device on which a user takes or uploads an image of themselves and simulates hairstyles and hair colors.

[0210] "Image input means" refers to technology that provides a camera function and a file upload function for users to take a photo of their face.

[0211] The "image processing means" is a technology that applies the selected hairstyle and hair color to a user's facial photograph and generates a simulation result.

[0212] "Display means" refers to technology that allows users to check the generated simulation results in the form of still images or videos.

[0213] "Information transmission means" refers to the technology that transmits the generated image and reservation information to the beauty salon.

[0214] The "hair salon display means" is a technology that allows the hair salon to display the received image and use it as a reference for treatment.

[0215] A "server" is a computer system that receives data sent from a user terminal and returns the results of analysis or simulation.

[0216] The "AI module" is a software component that uses artificial intelligence technology to analyze a user's image and apply hairstyles and hair colors.

[0217] The "interface means" refers to a user interface technology that allows the user to check the simulation results in real time and make fine adjustments as necessary.

[0218] This system allows users to simulate hairstyles and hair colors using a smartphone, smart glasses, or a head-mounted display on a photo of their face, and then share the results with a beauty salon. The specific system configuration and process are described below.

[0219] System Configuration

[0220] This system is broadly composed of a user terminal, a server, and a beauty salon terminal.

[0221] User Device

[0222] Image input means: Provides a camera function and file upload function for users to take a photo of their face.

[0223] Display means: Allows users to check the generated simulation results in still image or video format.

[0224] Information transmission means: The generated image and reservation information are transmitted to the server.

[0225] server

[0226] AI module: Receives images sent from the user's device and applies the selected hairstyle and hair color to the user's face photo.

[0227] Image processing means: Analyzes the user's facial photo and applies different hairstyles and hair colors.

[0228] Information transmission means: The generated image is sent back to the user's terminal and, if necessary, forwarded to the beauty salon.

[0229] Beauty salon terminal

[0230] Hair salon display means: The received image is displayed so that the stylist can use it as a reference for the treatment.

[0231] Hardware and software used

[0232] Hardware: The server is a computer system equipped with a high-performance CPU / GPU, and the user terminals are smartphones, smart glasses, and head-mounted displays.

[0233] Software: Python, FastAPI, OpenCV, deep learning frameworks (e.g. TensorFlow, PyTorch)

[0234] Data processing and calculation

[0235] The server receives the facial photo sent by the user and analyzes it using an AI module. It detects the facial area and applies the selected hairstyle and hair color to that area. The processed simulation results are sent back to the user's device, where the user can check and fine-tune in real time. If necessary, the simulation results can also be shared with the hair salon to help with the treatment.

[0236] Specific examples

[0237] For example, if a user wants to try a particular hairstyle, they open the smartphone app and take a photo of their face. Next, they select the desired style and color (e.g., bob cut + blue), and the app sends the image and selection information to the server. The server's AI module analyzes the image and generates a simulation result applying the selected style and color. The generated result is sent back to the user's device, where the user can review and fine-tune in real time. Once they are satisfied with the result, they can share it directly with the hair salon and complete the reservation.

[0238] Prompt Sentence Examples

[0239] "Apply a bob cut hairstyle and change the color to blue to the following photo of your face. Save the resulting image and send it back to the user."

[0240] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0241] Step 1:

[0242] Launch the app on the user's device and take or upload a photo of their face.

[0243] Input: A user-selected face photo.

[0244] Data processing: The app acquires a facial photo and converts it into a format that can be sent to the server.

[0245] Output: The converted facial photo data is sent to the server.

[0246] Step 2:

[0247] The server receives the facial photo sent from the user's device and analyzes it using an AI module.

[0248] Input: Facial photo data sent from the user device.

[0249] Data calculation: The AI ​​module analyzes the facial photo and detects the facial area.

[0250] Output: Face region detected data.

[0251] Step 3:

[0252] The user selects a hairstyle and color from a list provided within the app.

[0253] Input: User-selected hairstyle and color information.

[0254] Data processing: Send the selected information to the server.

[0255] Output: The selected hairstyle and color information is sent to the server.

[0256] Step 4:

[0257] Based on the facial photo and selection information received by the server, a simulation is performed to apply a hairstyle and color.

[0258] Input: Face region detected data and user selected hairstyle and color information.

[0259] Data calculation: The AI ​​module applies hairstyle and color to the face photo.

[0260] Output: Simulation results are generated.

[0261] Step 5:

[0262] The generated simulation results are returned to the user's terminal and displayed.

[0263] Input: Server-generated simulation results.

[0264] Data processing: Converting the generated simulation results into a displayable format.

[0265] Output: Simulation results sent to the user terminal.

[0266] Step 6:

[0267] The user checks the simulation results and makes fine adjustments as necessary.

[0268] Input: Simulation results sent back to the user.

[0269] What it does: Users can see the results in real time and make any necessary adjustments within the app.

[0270] Output: The final simulation result that you are satisfied with.

[0271] Step 7:

[0272] The user makes a reservation at a beauty salon based on the simulation results.

[0273] Input: Final satisfactory simulation results and reservation information.

[0274] Data processing: Simulation results and reservation information are sent to the server.

[0275] Output: Simulation results and reservation information are sent to the beauty salon terminal.

[0276] Step 8:

[0277] The salon terminal displays the received image for the stylist to check.

[0278] Input: Simulation results and reservation information sent from the server.

[0279] Specific operation: The salon terminal displays an image, which the stylist uses as a reference for the treatment.

[0280] Output: Satisfactory results based on the treatment are achieved.

[0281] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0282] The present invention relates to a system that allows users to simulate hairstyles and hair colors for their own images, recognizes the user's emotions, and makes optimal suggestions based on the results, which are then shared with beauty salons. In particular, the system provides a system that allows users to apply their preferred hairstyles and colors to their own facial photos, and based on the results of a motion simulation, allows beauty salons to more accurately implement treatments.

[0283] Overall system overview

[0284] This system is broadly composed of the following elements:

[0285] User Device

[0286] server

[0287] Beauty salon terminal

[0288] Emotion Engine

[0289] 1. User Device

[0290] The user terminal provides an interface for users to take or upload their own images and simulate hairstyles and hair colors. Specifically, the following functions are included:

[0291] Image input method: The user takes a photo of their face using the camera function or file upload function.

[0292] Display means: Simulation results are displayed to the user in still image or video format.

[0293] Information transmission means: The generated image and reservation information are sent to the beauty salon.

[0294] Emotion input means: Provides an interface for recognizing the user's emotions.

[0295] 2. Server

[0296] The server receives the images sent by the user and uses AI to apply hairstyles and colors, anonymize the images, analyze emotions, etc. It also forwards the generated images to the hair salon.

[0297] Image processing method: Apply different hairstyles and colors to the user's face photo and perform simulations.

[0298] Image processing method: To protect privacy, the user's face photo is anonymized.

[0299] Video generation means: Converts the simulation results from still images into video format, allowing users to check the motion simulation.

[0300] Information transmission means: The generated image and reservation information are sent to the beauty salon.

[0301] Emotion engine: Analyzes user emotions and records and manages emotional data.

[0302] 3. Beauty salon terminal

[0303] The salon terminal checks the image received from the server and projects it onto an AI mannequin, allowing the stylist to use it as a reference for the treatment.

[0304] Hair salon display means: The received image is displayed and used as a reference for treatment.

[0305] Specific examples

[0306] The program processing will be explained below using specific examples.

[0307] Example 1: Simulation flow

[0308] 1. User: Launches the app and takes or uploads a photo of their face.

[0309] The device sends the user's photo to the server.

[0310] 2. Server: The AI ​​module analyzes the received facial photo and cuts out the facial area.

[0311] 3. User: Select a hairstyle and color from the list provided within the app.

[0312] The terminal transmits the selection information to the server.

[0313] 4. Server: Apply the selected style and color to the user's image and generate the simulation result.

[0314] The generated image is sent back to the device.

[0315] 5. User: Check the simulation results using still images and videos, and make fine adjustments if necessary.

[0316] 6. User: Enter their feelings about the simulation results through the emotion input means.

[0317] 7. Server: The emotion engine analyzes the user's emotional data and, based on the results, re-suggests the most suitable hairstyle and color.

[0318] Example 2: Sharing information with a hair salon

[0319] 1. User: Makes a reservation at a beauty salon based on the simulation results.

[0320] The terminal transmits the reservation information and image data to the server.

[0321] 2. Server: Transfers the received reservation information and image data to the beauty salon terminal.

[0322] 3. Beauty salon terminal: The received image is projected onto the AI ​​mannequin and the stylist checks it.

[0323] 4. Beauty salon terminal: Based on the analysis results of the emotion engine, the terminal makes suggestions that will satisfy the user.

[0324] 5. Hair salon: The stylist performs the treatment based on the image and emotional data received.

[0325] The above is an embodiment of the present invention. The present invention facilitates communication between the user and the stylist, and realizes more accurate hair styling that takes the user's feelings into consideration.

[0326] The processing flow will be explained below.

[0327] Step 1:

[0328] A user launches the app and creates a new account or logs in with an existing account.

[0329] Step 2:

[0330] The user uses the camera function within the app to take a photo of their face or upload an existing photo, and the device sends this image data to the server.

[0331] Step 3:

[0332] The server analyzes the facial photos it receives using an AI module, identifies and cuts out the facial features, and stores the image data in a database after facial recognition processing is complete.

[0333] Step 4:

[0334] The user selects their preferred hairstyle and color from a list of hairstyle and color templates provided within the app, and the device sends the selection information to the server.

[0335] Step 5:

[0336] The server passes the selected hairstyle and color information to the AI ​​module, which applies it to the user's image, and returns the resulting image to the device.

[0337] Step 6:

[0338] The device displays the generated images to the user, or if the user wants to simulate the action, they request that the images be generated in animated format.

[0339] Step 7:

[0340] The server receives the request and generates the simulation results in the form of a video, which is then sent back to the device and displayed for the user to review.

[0341] Step 8:

[0342] The user uses the emotion input means based on the simulation result to input the emotion for the simulation result, and the terminal transmits the emotion data to the server.

[0343] Step 9:

[0344] The server analyzes the received emotion data using the emotion engine. Based on the analysis results, it suggests additional hairstyles and colors to increase the user's satisfaction. The results are then sent back to the device.

[0345] Step 10:

[0346] The device displays additional suggested hairstyles and colors to the user, who then selects a final style based on the suggestions.

[0347] Step 11:

[0348] If the user is satisfied with the simulation results, they make a reservation at the beauty salon. They enter the necessary information into the reservation form, attach the simulation results data, and submit it. The terminal then sends the reservation information and image data to the server.

[0349] Step 12:

[0350] The server receives the reservation information and image data and transfers them to the appropriate salon terminal.

[0351] Step 13:

[0352] The image data received by the salon terminal is projected onto an AI mannequin and displayed for the stylist to check.

[0353] Step 14:

[0354] At the salon, the stylist plans the combination of medications and treatment time based on the user's reservation information, the simulation results, and emotional data. The stylist then actually performs the treatment.

[0355] Step 15:

[0356] Users can check the results of the treatment and share feedback within the app if necessary. The device sends the feedback data to the server.

[0357] Example 2

[0358] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0359] Conventional hairstyle simulation systems were unable to provide optimal suggestions that took the user's emotions into account, resulting in frequent discrepancies between the simulation results and the actual results. Furthermore, there was insufficient sharing of images between the user and the hair salon, making it difficult to provide a highly satisfying treatment. There is a need for a method to share more accurate, emotion-based simulation results with the hair salon while maintaining the user's anonymity.

[0360] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0361] In this invention, the server includes image input means that allows a user to simulate hairstyles and hair colors for their own image, image processing means that generates images of the simulated hairstyles and hair colors, display means that allows the user to check the generated images, information transmission means that shares the generated images with the hair salon when making a reservation at the hair salon, display means that allows the hair salon to display and check the received images, emotion input means that recognizes the user's emotions, and an emotion engine that analyzes the user's emotions and makes optimal suggestions based on the emotion data. This makes it possible to suggest optimal hairstyles and hair colors based on the user's emotions, reduces the gap between the simulation results and the actual treatment results, and enables information to be shared effectively with the hair salon while maintaining anonymity.

[0362] "Image input means" refers to a device or function that allows a user to take or upload their own image and input it into the system.

[0363] "Image processing means" refers to a function for applying and generating simulated hairstyles and hair colors to a user's image.

[0364] A "display means" is a device or function for visually presenting the generated image to a user.

[0365] The "information transmission means" is a communication means for sharing the generated image and reservation information with the beauty salon.

[0366] The "emotion input means" is an interface for recognizing the user's emotions and inputting them into the system.

[0367] The "emotion engine" is a function that analyzes the user's emotional data and makes optimal suggestions based on that data.

[0368] "Anonymization processing" is the process of anonymizing a facial photo to protect the user's privacy.

[0369] The "moving image generating means" is a function for converting the image of the simulated hairstyle and hair color from a still image to a moving image format.

[0370] The "beauty salon display means" is a device or function that displays the received image at the beauty salon and serves as a reference for the treatment.

[0371] This invention relates to a system that allows users to simulate hairstyles and hair colors for their own images, recognizes the user's emotions, and makes optimal suggestions based on the results, which are then shared with a hair salon. This system allows users to apply their favorite hairstyles and colors to their own facial photos and check the results as a motion simulation. Furthermore, by sharing the simulation results with the hair salon, the treatment can be carried out more accurately.

[0372] Overall system overview

[0373] This system mainly consists of the following elements:

[0374] User Device

[0375] server

[0376] Beauty salon terminal

[0377] Emotion Engine

[0378] User Device

[0379] The user device provides an interface for users to take or upload their own images and simulate hairstyles and hair colors. Specifically, it includes the following functions:

[0380] Image input method: The user takes a photo of their face using the camera function or file upload function.

[0381] Display means: Simulation results are displayed to the user in still image or video format.

[0382] Information transmission means: The generated image and reservation information are sent to the beauty salon.

[0383] Emotion input means: Provides an interface for recognizing the user's emotions.

[0384] server

[0385] The server receives the images sent by the user and uses AI to apply hairstyles and colors, anonymize the images, analyze emotions, etc. It also forwards the generated images to the hair salon.

[0386] Image processing method: Apply different hairstyles and colors to the user's face photo and perform simulations.

[0387] Image processing method: To protect privacy, the user's face photo is anonymized.

[0388] Video generation means: Converts the simulation results from still images into video format, allowing users to check the motion simulation.

[0389] Information transmission means: The generated image and reservation information are sent to the beauty salon.

[0390] Emotion engine: Analyzes user emotions and records and manages emotional data.

[0391] Beauty salon terminal

[0392] The salon terminal checks the image received from the server and projects it onto an AI mannequin, allowing the stylist to use it as a reference for the treatment.

[0393] Hair salon display means: The received image is displayed and used as a reference for treatment.

[0394] Specific examples

[0395] The processing of the system will be explained in detail below using specific examples.

[0396] Example 1: Simulation flow

[0397] The user launches the app and takes a photo of their face using their smartphone's camera. This image is then sent from the user's device to the server. The server then uses an AI module to analyze the received photo and extract the facial features. This process is performed using Python and OpenCV. The user selects a hairstyle and color, such as a bob cut and red, from a list within the app, and sends this information from the user's device to the server.

[0398] The server applies the received style and color to the user's image and generates a simulation result. TensorFlow and Keras are used for this. The generated simulation results are generated in the form of still images or videos and sent from the server to the user's device. The user can check the simulation results on the app and make fine adjustments as needed.

[0399] Example 2: Sharing information with a hair salon

[0400] The user makes a reservation at the hair salon based on the final simulation results. This reservation information and simulation image are sent from the user's device to the server. The server then forwards the received reservation information and image data to the hair salon's device. The received simulation results are projected onto the AI ​​mannequin at the hair salon's device, where the stylist can check them.

[0401] Prompt Sentence Examples

[0402] "Use the hairstyle simulation app to apply different styles and colors to your face photo and see the simulation results. Then, tell us how you feel about the results."

[0403] "Book a salon appointment based on the hairstyle simulation generated by the app and share the results with your salon."

[0404] This system facilitates communication between the user and the stylist, enabling more accurate hair styling that takes the user's emotions into account.

[0405] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0406] Step 1: Image Input

[0407] The user launches the app. The user takes or selects a photo of their face using the device's camera function or file upload function. The input is the user's photo of their face, and the output is image data stored on the device. Specifically, the smartphone's camera app launches, and the user takes a photo of their face.

[0408] Step 2: Send image

[0409] The device sends the acquired facial photo to the server. The input is image data on the device, and the output is image data transferred to the server. Specifically, the image data is uploaded to the server via the Internet.

[0410] Step 3: Face Recognition and Segmentation

[0411] The AI ​​module analyzes the image received by the server and extracts the facial area. The input is a face photo data, and the output is image data of the extracted facial area. Specifically, it uses Python's OpenCV library to execute a face detection algorithm and identify the facial area.

[0412] Step 4: Choose your hairstyle and color

[0413] The user selects a hairstyle and hair color from options within the app. The input is the user's selection, and the output is the selected style and color information. Specifically, the user taps to select the desired style and color from the thumbnails on the app screen.

[0414] Step 5: Send your selection

[0415] The terminal sends the selected style and color information to the server. The input is the selection information, and the output is the style and color information transferred to the server. Specifically, the selection data is sent to the server via the Internet.

[0416] Step 6: Run the simulation

[0417] The server applies the selected style and color to the user's image and generates a simulation result. The input is the image data of the extracted face area and style and color information, and the output is the generated simulation result image or video data. Specifically, the style and color application process is performed using TensorFlow and Keras.

[0418] Step 7: Send simulation results

[0419] The server returns the generated simulation results to the terminal. The input is the simulation result data, and the output is the simulation result transferred to the terminal. Specifically, the simulation results are sent to the user terminal via the Internet.

[0420] Step 8: Check the simulation results and make adjustments

[0421] The user checks the simulation results and makes fine adjustments as necessary. The input is image or video data of the simulation results, and the output is the final simulation result after adjustments. Specifically, the user views the simulation results on the app screen and makes fine adjustments to the style and color.

[0422] Step 9: Enter emotions

[0423] The user inputs their feelings about the simulation results. The input is emotional information, and the output is emotional data recorded on the device. Specifically, the user selects buttons provided on the app, such as "Satisfied," "Neutral," or "Dissatisfied."

[0424] Step 10: Sending Emotion Data

[0425] The device sends emotional data to the server. The input is the emotional data on the device, and the output is the emotional data transferred to the server. Specifically, the emotional information is sent to the server via the Internet.

[0426] Step 11: Sentiment analysis and result suggestions

[0427] The server's emotion engine analyzes the emotion data and, based on the results, makes new recommendations for optimal hairstyles and colors. The input is emotion data, and the output is recommendation information based on the analysis results. Specifically, emotion analysis is performed using the IBM Watson Emotion Analysis API to generate optimal recommendations.

[0428] Step 12: Make a reservation at the salon and submit the information

[0429] The user makes a reservation at a hair salon based on the final simulation results. The input is the reservation information and the final simulation results, and the output is the reservation information and image data sent to the server. Specifically, the user selects the desired date and time and hair salon on the reservation page within the app, and that information is sent to the server.

[0430] Step 13: Transfer to salon device

[0431] The server transfers the received reservation information and image data to the salon terminal. The input is the reservation information and image data, and the output is the information sent to the salon terminal. Specifically, the data is transferred to the salon terminal using a backend service based on Java Spring Boot.

[0432] Step 14: Projecting onto the AI ​​mannequin

[0433] The salon terminal receives the simulation results and projects them onto an AI mannequin for the stylist to check. The input is the simulation result data, and the output is the simulated image projected onto the AI ​​mannequin. Specifically, a dedicated display and AI mannequin are used to project the simulation results.

[0434] Step 15: Treatment proposal based on emotion analysis results

[0435] Based on the analysis results of the emotion engine, the salon terminal will propose treatments that will satisfy the user. The input is the emotion analysis results, and the output is treatment proposal information. Specifically, the stylist will provide the user with the optimal treatment plan while referring to the emotion analysis results.

[0436] Step 16: Treatment

[0437] The stylist at the hair salon performs the treatment based on the received image and emotional data. The input is the final simulation image and emotional data, and the output is the hairstyle that will be applied to the user. Specifically, the AI ​​performs the actions of cutting and coloring the user's hair based on the simulation on the mannequin.

[0438] (Application example 2)

[0439] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0440] In conventional hairstyle and hair color simulation systems, it was difficult for users to not only simulate hairstyles but also receive appropriate suggestions based on their emotions. Furthermore, there were no examples of virtual stores where users could easily share simulation results and then immediately purchase or make reservations. This meant that users could not fully confirm the satisfaction and suitability of the product before actually purchasing, potentially resulting in a dissatisfying experience.

[0441] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0442] In this invention, the server includes an image input means that allows a user to simulate hairstyles and hair colors for an image of the user, an image processing means that generates an image of the simulated hairstyle and hair color, a display means that allows the user to check the generated image, an information transmission means that shares the generated image with the hair salon when making a reservation at the hair salon, a display means that allows the hair salon to display and check the received image, an emotion analysis means that analyzes the user's emotions and makes optimal suggestions based on the results, an image regeneration means that regenerates an optimal simulation based on the user's emotion data, and a purchase linkage means that allows the user to purchase items that the user likes based on the try-on simulation results. This allows the user to receive appropriate suggestions that match their emotions based on the simulation results, and further allows them to try on simulated items in a virtual store and directly purchase their favorite items.

[0443] "Image input means" means a device or interface that allows a user to take or upload an image of themselves.

[0444] "Image processing means" refers to algorithms or software for applying and simulating hairstyles and hair colors to photographed or uploaded images.

[0445] "Display means" refers to a device or screen that shows the generated simulation image to the user.

[0446] The "information transmission means" is a communication means for transmitting the generated simulation image to a beauty salon or other service provider.

[0447] An "emotion analysis method" is an algorithm or software that analyzes a user's emotions and makes optimal suggestions based on the results.

[0448] The "image regeneration means" is an algorithm or software for regenerating the simulation based on the results of the emotion analysis.

[0449] The "purchase linking method" is a function that allows users to purchase items they like on the spot based on the results of a try-on simulation.

[0450] "Image processing means" refers to algorithms or software used to process a user's image for anonymity.

[0451] The "video generation means" is an algorithm or software for generating a simulation image in video format so that the user can check the operation simulation.

[0452] A "virtual try-on means" is an interface or software that allows a user to simulate trying on clothes using a photograph of the user's face or whole body.

[0453] To implement this invention, it is necessary to build a system in which the user terminal, server, virtual store system, and beauty salon terminal all function in cooperation with each other. This system is composed of the following elements:

[0454] User Device

[0455] User terminals include smartphones, smart glasses, head-mounted displays, etc. They have the following functions:

[0456] Image input method: An interface that allows a user to take or upload a face or full-body photo, for example by selecting an image from a smartphone camera or photo gallery.

[0457] Display means: A screen for displaying simulation results and proposals to users. Simulation images and videos can be viewed.

[0458] Emotion input means: An interface for analyzing user emotions. Uses an AI engine (e.g., IBM Watson, Microsoft Azure Emotion API).

[0459] server

[0460] The server has the necessary hardware and software to perform the following functions:

[0461] Image processing means: Algorithms (e.g., OpenCV, TensorFlow) for applying hairstyles and hair colors to user-submitted images.

[0462] Image processing methods: Algorithms used to anonymize user images (e.g., image blurring or masking).

[0463] Video generation means: Software for converting simulation results into video format and providing operational simulation.

[0464] Sentiment analysis means: Equipped with a function to analyze emotional data and optimize suggestions (e.g., IBM Watson, Microsoft Azure).

[0465] Image regeneration means: An algorithm that regenerates the simulation based on the results of emotion analysis.

[0466] Purchase linkage: A feature that allows users to purchase items they like on the spot.

[0467] The server analyzes the image data received from the user and performs the following data processing and calculations:

[0468] Image analysis: Identify facial and body features using an AI engine (e.g., TensorFlow).

[0469] Apply Hairstyle: Apply the selected hairstyle and hair color to the image.

[0470] Sentiment Analysis: Analyzes emotional data using a sentiment analysis engine to generate optimal suggestions.

[0471] Simulation Regeneration: Regenerate the simulation based on the analysis results.

[0472] Display of simulation results: Generated images and videos are sent back to the user's device.

[0473] Virtual store system

[0474] The virtual store system implements the following functions:

[0475] Virtual try-on tool: An interface for simulating trying on clothes using a photo of the user's face or full body.

[0476] Purchase linking method: Provide users with a link to purchase items they like based on the simulation results.

[0477] Beauty salon terminal

[0478] The salon terminal includes computers and tablets used by salons. It has the following functions:

[0479] Display means: Simulation images and videos received from the server are displayed and used as a reference for treatment.

[0480] Information transmission method: A function for checking reservation information and simulation results and reflecting them in treatment.

[0481] Specific examples

[0482] The processing flow will be explained below with specific examples.

[0483] 1. Image acquisition: Take a photo of your face with your smartphone and upload it to the app.

[0484] 2. Image analysis: Images sent to the server are analyzed using an AI engine (TensorFlow).

[0485] 3. Simulation Generation: Apply the selected hairstyle and color to the image and generate the result.

[0486] 4. Sentiment analysis: The user inputs their feelings about the generated image, which is then analyzed by the sentiment analysis engine.

[0487] 5. Regeneration: Based on the results of sentiment analysis, the optimal simulation is regenerated.

[0488] 6. Result display: Display the simulation results to the user and readjust if necessary.

[0489] 7. Purchasing integration: Try on items in a virtual store system and purchase your favorite items on the spot.

[0490] Prompt Sentence Examples

[0491] Example prompts for generative AI models:

[0492] "Please simulate applying items selected from the clothing list below to a face photo taken or uploaded by the user. Please combine the items from the provided list with the user's image and return the results. Furthermore, please analyze the user's emotional data and suggest optimal items based on their emotions. Please generate a final image containing the suggested results."

[0493] In this way, the present invention provides a comprehensive system that allows users to simulate hairstyles and fashions that are more suitable for them, and by adding emotion analysis and virtual try-on functions, it is possible to increase user satisfaction.

[0494] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0495] Step 1:

[0496] Users launch the app using a smartphone, smart glasses, or head-mounted display and take or upload a photo of their face or full body.

[0497] Input: A face or full-body photo of the user.

[0498] Output: Image data captured or uploaded.

[0499] Specific behavior: Launch the camera app and take a photo, or select an image from your photo gallery.

[0500] Step 2:

[0501] The user terminal transmits the captured or uploaded image data to the server.

[0502] Input: Image data captured or uploaded by the user.

[0503] Output: Image data sent to the server.

[0504] What it does: Encodes image data and uploads it to a server via a secure communication protocol.

[0505] Step 3:

[0506] The server analyzes the received image data using an AI engine (e.g., TensorFlow) to identify facial and physical features.

[0507] Input: Image data sent to the server.

[0508] Output: Analyzed facial and body feature data.

[0509] How it works: Image data is fed into the AI ​​model, feature points are extracted, and facial recognition algorithms are applied to identify facial contours and features.

[0510] Step 4:

[0511] The server applies the user's selected hairstyle and color to the image to generate a simulation.

[0512] Input: Analyzed facial and body feature data and user-selected hairstyle and color information.

[0513] Output: Image data of the simulation results.

[0514] What it does: Based on facial feature points, it applies the selected hairstyle and color to generate a realistic simulation image.

[0515] Step 5:

[0516] The server transmits the generated simulation image to the user terminal and displays it.

[0517] Input: Image data of the simulation results.

[0518] Output: The simulated image displayed on the user's device.

[0519] Specific operation: Encode the simulated image and send it to the user's device, where it is decoded and displayed on the screen.

[0520] Step 6:

[0521] The user checks the simulation results and inputs his / her feelings about the results using the emotion input means.

[0522] Input: Simulation result display and user emotion data.

[0523] Output: The input information, including emotion data.

[0524] Specific behavior: Look at the simulation results, make decisions, and operate the interface for emotion input.

[0525] Step 7:

[0526] The server analyzes the emotional data using an emotion analysis engine (e.g., IBM Watson) and generates optimal suggestions.

[0527] Input: User emotion data.

[0528] Output: Data with the best suggestions.

[0529] What it does: Passes input data to the sentiment analysis engine, retrieves the analysis results, and generates appropriate suggestions.

[0530] Step 8:

[0531] The server regenerates the simulation based on the emotion analysis results and transmits it again to the user terminal.

[0532] Input: Sentiment analysis results and original simulated image data.

[0533] Output: Regenerated simulated image data.

[0534] Specific operation: A new simulation is generated using hairstyles and colors adjusted based on the results of emotion analysis and transmitted to the user's device.

[0535] Step 9:

[0536] Users can simulate trying on items using a virtual fitting system and purchase the items they like on the spot.

[0537] Input: Regenerated simulation image and user selection information.

[0538] Output: Information about the completed purchase.

[0539] Specific actions: Operate the try-on simulation in the virtual try-on system and use the interface to complete the purchase process.

[0540] The above is a specific processing flow for carrying out the present invention.

[0541] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0542] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0543] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0544] [Second embodiment]

[0545] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0546] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0547] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0548] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0549] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0550] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0551] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0552] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0553] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0554] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0555] In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0556] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0557] The present invention relates to a system that allows users to simulate hairstyles and hair colors for their own images and share the results with beauty salons. In particular, the system provides a system that allows users to apply their preferred hairstyles and colors to their own facial photos and perform a simulation based on the results of the simulation, enabling more accurate treatments at beauty salons.

[0558] Overall system overview

[0559] This system is broadly composed of the following elements:

[0560] User Device

[0561] server

[0562] Beauty salon terminal

[0563] 1. User Device

[0564] The user terminal provides an interface for users to take or upload their own images and simulate hairstyles and hair colors. Specifically, the following functions are included:

[0565] Image input method: The user takes a photo of their face using the camera function or file upload function.

[0566] Display means: Simulation results are displayed to the user in still image or video format.

[0567] Information transmission means: The generated image and reservation information are sent to the beauty salon.

[0568] 2. Server

[0569] The server receives the image sent by the user and uses AI to apply hairstyles, colors, anonymize, etc. It also forwards the generated image to the hair salon.

[0570] Image processing method: Apply different hairstyles and colors to the user's face photo and perform simulations.

[0571] Image processing method: To protect privacy, the user's face photo is anonymized.

[0572] Video generation means: Converts the simulation results from still images into video format, allowing users to check the motion simulation.

[0573] Information transmission means: The generated image and reservation information are sent to the beauty salon.

[0574] 3. Beauty salon terminal

[0575] The salon terminal checks the image received from the server and projects it onto an AI mannequin, allowing the stylist to use it as a reference for the treatment.

[0576] Hair salon display means: The received image is displayed and used as a reference for treatment.

[0577] Specific examples

[0578] The program processing will be explained below using specific examples.

[0579] Example 1: Simulation flow

[0580] 1. User: Launches the app and takes or uploads a photo of their face.

[0581] The device sends the user's photo to the server.

[0582] 2. Server: The AI ​​module analyzes the received facial photo and cuts out the facial area.

[0583] 3. User: Select a hairstyle and color from the list provided within the app.

[0584] The terminal transmits the selection information to the server.

[0585] 4. Server: Apply the selected style and color to the user's image and generate the simulation result.

[0586] The generated image is sent back to the device.

[0587] 5. User: Check the simulation results using still images and videos, and make fine adjustments if necessary.

[0588] Example 2: Sharing information with a hair salon

[0589] 1. User: Makes a reservation at a beauty salon based on the simulation results.

[0590] The terminal transmits the reservation information and image data to the server.

[0591] 2. Server: Transfers the received reservation information and image data to the beauty salon terminal.

[0592] 3. Beauty salon terminal: The received image is projected onto the AI ​​mannequin and the stylist checks it.

[0593] 4. Hair salon: The stylist performs the treatment based on the image received.

[0594] The above is an embodiment of the present invention. The present invention facilitates communication between the user and the stylist, and allows for more accurate hair styling.

[0595] The processing flow will be explained below.

[0596] Step 1:

[0597] A user launches the app and creates a new account or logs in with an existing account.

[0598] Step 2:

[0599] The user uses the camera function within the app to take a photo of their face or upload an existing photo, and the device sends this image data to the server.

[0600] Step 3:

[0601] The server analyzes the facial photos it receives using an AI module, identifies and cuts out the facial features, and stores the image data in a database after facial recognition processing is complete.

[0602] Step 4:

[0603] The user selects their preferred hairstyle and color from a list of hairstyle and color templates provided within the app, and the device sends the selection information to the server.

[0604] Step 5:

[0605] The server passes the selected hairstyle and color information to the AI ​​module, which applies it to the user's image, and returns the resulting image to the device.

[0606] Step 6:

[0607] The device displays the generated images to the user, or if the user wants to simulate the action, they request that the images be generated in animated format.

[0608] Step 7:

[0609] The server receives the request and generates the simulation results in the form of a video, which is then sent back to the device and displayed for the user to review.

[0610] Step 8:

[0611] If the user is satisfied with the simulation results, they make a reservation at the beauty salon. They enter the necessary information into the reservation form, attach the simulation results data, and submit it. The terminal then sends the reservation information and image data to the server.

[0612] Step 9:

[0613] The server receives the reservation information and image data and transfers them to the appropriate salon terminal.

[0614] Step 10:

[0615] The image data received by the salon terminal is projected onto an AI mannequin and displayed for the stylist to check.

[0616] Step 11:

[0617] At the salon, the stylist plans the combination of medications and treatment time based on the user's reservation information and the simulation results, and then actually performs the treatment.

[0618] Step 12:

[0619] Users can check the results of the treatment and share feedback within the app if necessary.

[0620] Example 1

[0621] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0622] When users receive treatment at a beauty salon, it is difficult for them to accurately communicate their desired hairstyle and hair color. Furthermore, when users simulate hairstyles and hair colors using photos of their own faces, it is difficult to get an idea from still images alone, and privacy is not protected if the images are not anonymized. Since it is not possible to simulate changes in color due to the seasons or weather, the results may not be as expected.

[0623] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0624] In this invention, the server includes a means for projecting the received image onto the AI ​​mannequin, a face detection means for extracting the user's face, and a video generation means for viewing the simulation results in video format. This allows the user to accurately communicate their desired hairstyle and hair color, and viewing the simulation results in video makes it easier to understand the image, while also protecting privacy through anonymity. Furthermore, by simulating color changes due to seasons and weather, it is possible to achieve the expected results.

[0625] "Image input means" refers to the means used by a user to take or upload a photo of their face.

[0626] The "image processing means" is a means for applying a hairstyle and hair color to a facial photo sent by a user and performing a simulation.

[0627] The "display means" is a means for displaying the generated simulation results so that the user can check them.

[0628] The "information transmission means" is a means for transmitting the generated simulation image to the beauty salon.

[0629] "Means for projecting the received image onto the AI ​​mannequin" refers to the means by which the beauty salon projects the received image onto the AI ​​mannequin and visually confirms it.

[0630] The "face detection means" is a means for detecting and extracting the user's face, and mainly uses a deep learning model.

[0631] The "moving image generating means" is a means for generating simulation results in the form of not only still images but also moving images, allowing the user to check the operation simulation.

[0632] "Image processing means" refers to a means of anonymizing a user's facial photograph to protect their privacy.

[0633] The "data compression means" is a means for compressing the generated image data to improve communication efficiency.

[0634] The "function to simulate color changes due to seasons and weather" is a function for simulating changes in hair color due to environmental conditions.

[0635] This system allows users to simulate hairstyles and hair colors on their own images and share them with hair salons. This allows users to check their desired style in advance and ensure smooth and accurate treatment at the salon. The system consists of three main components: a user terminal, a server, and a salon terminal.

[0636] User Device

[0637] The user device provides an interface that allows users to take or upload their own images and simulate hairstyles and hair colors. Specifically, it has the following functions. First, as an image input means, the camera function or file upload function of a smartphone or tablet is used. The user can launch the app and take or upload a photo of their face. Next, as a display means, the simulation results are displayed to the user in still image or video format. This allows the user to visually check various hairstyles and color variations. Finally, as an information transmission means, the generated simulation image and reservation information can be sent to the hair salon.

[0638] server

[0639] The server receives images sent by users and uses AI to apply hairstyles and colors, and anonymize them. It also has the function of transmitting the generated images to the hair salon. Specifically, the functions are as follows: First, as an image processing method, an AI module (e.g., a deep learning model using TensorFlow or PyTorch) is used to analyze the user's facial photo and cut out the facial area. Next, as an image processing method, anonymization is performed on the user's facial photo to protect privacy. Finally, as a video generation method, the simulation results of still images are converted into video format so that the user can check the movement simulation. Finally, as an information transmission method, the generated images and reservation information are sent to the hair salon.

[0640] Beauty salon terminal

[0641] The salon terminal checks the image received from the server and projects it onto an AI mannequin, allowing the stylist to use it as a reference for the treatment. Specifically, it has the following functions. First, as a salon display means, it displays the received image using a display or projector, allowing the stylist to check the treatment details. Furthermore, by projecting the received image onto the AI ​​mannequin, the style can be visually checked in real time.

[0642] Specific examples

[0643] Example 1: Simulation flow

[0644] 1. User: Launches the app and takes or uploads a photo of their face.

[0645] The device sends the user's photo to the server.

[0646] 2. Server: The received face photo is analyzed using an AI module and facial features are extracted. For example, modules using OpenCV or deep learning are used.

[0647] 3. User: Select a hairstyle and color from the list provided within the app.

[0648] The terminal transmits the selection information to the server.

[0649] 4. Server: Apply the selected style and color to the user's image and generate the simulation result.

[0650] The generated image is sent back to the device.

[0651] 5. User: Check the simulation results using still images and videos, and make fine adjustments if necessary.

[0652] Example prompt for a generative AI model:

[0653] Input image: A face photo uploaded by the user

[0654] Hairstyle: Classic Bob

[0655] Hair Color: Blonde

[0656] Output: Generates and returns a simulated image with the specified hairstyle and hair color.

[0657] Example 2: Sharing information with a hair salon

[0658] 1. User: Makes a reservation at a beauty salon based on the simulation results.

[0659] The terminal transmits the reservation information and image data to the server.

[0660] 2. Server: Transfers the received reservation information and image data to the beauty salon terminal.

[0661] 3. Beauty salon terminal: The received image is projected onto the AI ​​mannequin and the stylist checks it.

[0662] 4. Hair salon: The stylist performs the treatment based on the image received.

[0663] This allows users to efficiently simulate their desired hairstyle and color, facilitating smooth communication with the hairdresser. By using this system, users can check their appearance in advance, resulting in a more satisfying experience at the hairdresser.

[0664] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0665] Step 1:

[0666] User: Launches the app and takes or uploads a photo of their face.

[0667] Specific actions include taking a photo of your face using the camera on your smartphone or tablet, or selecting and uploading an existing image file from a folder.

[0668] Input: User's face photo (image file)

[0669] Output: Facial photo data is saved on the user's device.

[0670] Step 2:

[0671] Terminal: The acquired facial photo is compressed and sent to a server via the Internet.

[0672] Specifically, the image file is encoded in JPEG format and sent to the server via an HTTP POST request.

[0673] Input: Facial photo data (image file)

[0674] Output: Compressed face photo data is sent to the server.

[0675] Step 3:

[0676] Server: The received image is analyzed using an AI module, and facial features are detected and extracted.

[0677] Specifically, it applies a facial recognition algorithm using deep learning libraries such as OpenCV and TensorFlow to identify and cut out the facial area.

[0678] Input: Compressed face photo data

[0679] Output: Image data of the cut-out face area

[0680] Step 4:

[0681] User: Selects a hairstyle and color from a list provided through the in-app interface.

[0682] Specifically, you click on the drop-down menus and icons displayed within the app to select the desired hairstyle and color.

[0683] Input: User's selection information (hair style and color)

[0684] Output: The selected hairstyle and color information is saved on the user's device.

[0685] Step 5:

[0686] Terminal: Sends the selection information to the server.

[0687] Specifically, the selected hairstyle and color information is encoded in JSON format and sent to the server via an HTTP POST request.

[0688] Input: User's selection information (hair style and color)

[0689] Output: The selection is sent to the server.

[0690] Step 6:

[0691] Server: Based on the selection information, apply the simulation to the user's face photo.

[0692] Specifically, it uses generative AI models such as TensorFlow and PyTorch to apply hairstyles and colors to a user's image, generating a realistic simulation.

[0693] Input: Image data of the cut-out face, selection information (hairstyle and color)

[0694] Output: Image data of the simulation results

[0695] Example prompt for a generative AI model:

[0696] Input image: A face photo uploaded by the user

[0697] Hairstyle: Classic Bob

[0698] Hair Color: Blonde

[0699] Output: Generates and returns a simulated image with the specified hairstyle and hair color.

[0700] Step 7:

[0701] Server: Generates simulation results in video format and sends them back to the user's device.

[0702] Specifically, the simulation results of still images are converted into video format (MP4, etc.) and sent to the user terminal as an HTTP response.

[0703] Input: Still image data of simulation results

[0704] Output: The simulation result video data is sent to the user's device.

[0705] Step 8:

[0706] User: Review the returned simulation results and make any necessary adjustments.

[0707] Specifically, you can change the hairstyle or color again within the app, start the simulation again, and confirm the final result.

[0708] Input: Simulation result video data

[0709] Output: Final simulation result data

[0710] Step 9:

[0711] User: Sends the final simulation results and reservation information to the hair salon.

[0712] Specifically, the reservation information (date and time, desired treatment) and a simulation image are sent to the server.

[0713] Input: Final simulation result data, reservation information

[0714] Output: The final simulation results and reservation information are sent to the server.

[0715] Step 10:

[0716] Server: Transfers the received reservation information and image data to the beauty salon terminal.

[0717] Specifically, the reservation information and the simulation image are sent to the beauty salon terminal.

[0718] Input: Final simulation results and reservation information

[0719] Output: Reservation information and image data are sent to the salon terminal.

[0720] Step 11:

[0721] Beauty salon terminal: The received image data is projected onto an AI mannequin for the stylist to check.

[0722] Specifically, the received image is displayed using a display or projector.

[0723] Input: Image data, reservation information

[0724] Output: Treatment is performed based on the projected image data.

[0725] The above are the specific processing steps of the system, which are a series of steps to simulate the hairstyle and hair color desired by the user and reflect them in the treatment at the beauty salon.

[0726] (Application example 1)

[0727] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0728] Conventional hair simulation systems have difficulty in sharing simulation results between users and hair salons, which can lead to misunderstandings during treatment. It is also difficult to check and fine-tune simulation results in real time, making it difficult for users to accurately communicate their desired style.

[0729] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0730] In this invention, the server includes means for receiving a facial photo taken or uploaded by a user terminal and analyzing it with an AI module, means for applying a selected hairstyle and hair color to the user's facial photo and returning the generated image, and an AI module for receiving the user's facial photo and generating a hairstyle and color simulation result in real time, allowing the user to simulate a hairstyle and hair color for their own image, check the result in real time, and share it with a hair salon.

[0731] "User terminal" means a device on which a user takes or uploads an image of themselves and simulates hairstyles and hair colors.

[0732] "Image input means" refers to technology that provides a camera function and a file upload function for users to take a photo of their face.

[0733] The "image processing means" is a technology that applies the selected hairstyle and hair color to a user's facial photograph and generates a simulation result.

[0734] "Display means" refers to technology that allows users to check the generated simulation results in the form of still images or videos.

[0735] "Information transmission means" refers to the technology that transmits the generated image and reservation information to the beauty salon.

[0736] The "hair salon display means" is a technology that allows the hair salon to display the received image and use it as a reference for treatment.

[0737] A "server" is a computer system that receives data sent from a user terminal and returns the results of analysis or simulation.

[0738] The "AI module" is a software component that uses artificial intelligence technology to analyze a user's image and apply hairstyles and hair colors.

[0739] The "interface means" refers to a user interface technology that allows the user to check the simulation results in real time and make fine adjustments as necessary.

[0740] This system allows users to simulate hairstyles and hair colors using a smartphone, smart glasses, or a head-mounted display on a photo of their face, and then share the results with a beauty salon. The specific system configuration and process are described below.

[0741] System Configuration

[0742] This system is broadly composed of a user terminal, a server, and a beauty salon terminal.

[0743] User Device

[0744] Image input means: Provides a camera function and file upload function for users to take a photo of their face.

[0745] Display means: Allows users to check the generated simulation results in still image or video format.

[0746] Information transmission means: The generated image and reservation information are transmitted to the server.

[0747] server

[0748] AI module: Receives images sent from the user's device and applies the selected hairstyle and hair color to the user's face photo.

[0749] Image processing means: Analyzes the user's facial photo and applies different hairstyles and hair colors.

[0750] Information transmission means: The generated image is sent back to the user's terminal and, if necessary, forwarded to the beauty salon.

[0751] Beauty salon terminal

[0752] Hair salon display means: The received image is displayed so that the stylist can use it as a reference for the treatment.

[0753] Hardware and software used

[0754] Hardware: The server is a computer system equipped with a high-performance CPU / GPU, and the user terminals are smartphones, smart glasses, and head-mounted displays.

[0755] Software: Python, FastAPI, OpenCV, deep learning frameworks (e.g. TensorFlow, PyTorch)

[0756] Data processing and calculation

[0757] The server receives the facial photo sent by the user and analyzes it using an AI module. It detects the facial area and applies the selected hairstyle and hair color to that area. The processed simulation results are sent back to the user's device, where the user can check and fine-tune in real time. If necessary, the simulation results can also be shared with the hair salon to help with the treatment.

[0758] Specific examples

[0759] For example, if a user wants to try a particular hairstyle, they open the smartphone app and take a photo of their face. Next, they select the desired style and color (e.g., bob cut + blue), and the app sends the image and selection information to the server. The server's AI module analyzes the image and generates a simulation result applying the selected style and color. The generated result is sent back to the user's device, where the user can review and fine-tune in real time. Once they are satisfied with the result, they can share it directly with the hair salon and complete the reservation.

[0760] Prompt Sentence Examples

[0761] "Apply a bob cut hairstyle and change the color to blue to the following photo of your face. Save the resulting image and send it back to the user."

[0762] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0763] Step 1:

[0764] Launch the app on the user's device and take or upload a photo of their face.

[0765] Input: A user-selected face photo.

[0766] Data processing: The app acquires a facial photo and converts it into a format that can be sent to the server.

[0767] Output: The converted facial photo data is sent to the server.

[0768] Step 2:

[0769] The server receives the facial photo sent from the user's device and analyzes it using an AI module.

[0770] Input: Facial photo data sent from the user device.

[0771] Data calculation: The AI ​​module analyzes the facial photo and detects the facial area.

[0772] Output: Face region detected data.

[0773] Step 3:

[0774] The user selects a hairstyle and color from a list provided within the app.

[0775] Input: User-selected hairstyle and color information.

[0776] Data processing: Send the selected information to the server.

[0777] Output: The selected hairstyle and color information is sent to the server.

[0778] Step 4:

[0779] Based on the facial photo and selection information received by the server, a simulation is performed to apply a hairstyle and color.

[0780] Input: Face region detected data and user selected hairstyle and color information.

[0781] Data calculation: The AI ​​module applies hairstyle and color to the face photo.

[0782] Output: Simulation results are generated.

[0783] Step 5:

[0784] The generated simulation results are returned to the user's terminal and displayed.

[0785] Input: Server-generated simulation results.

[0786] Data processing: Converting the generated simulation results into a displayable format.

[0787] Output: Simulation results sent to the user terminal.

[0788] Step 6:

[0789] The user checks the simulation results and makes fine adjustments as necessary.

[0790] Input: Simulation results sent back to the user.

[0791] What it does: Users can see the results in real time and make any necessary adjustments within the app.

[0792] Output: The final simulation result that you are satisfied with.

[0793] Step 7:

[0794] The user makes a reservation at a beauty salon based on the simulation results.

[0795] Input: Final satisfactory simulation results and reservation information.

[0796] Data processing: Simulation results and reservation information are sent to the server.

[0797] Output: Simulation results and reservation information are sent to the beauty salon terminal.

[0798] Step 8:

[0799] The salon terminal displays the received image for the stylist to check.

[0800] Input: Simulation results and reservation information sent from the server.

[0801] Specific operation: The salon terminal displays an image, which the stylist uses as a reference for the treatment.

[0802] Output: Satisfactory results based on the treatment are achieved.

[0803] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0804] The present invention relates to a system that allows users to simulate hairstyles and hair colors for their own images, recognizes the user's emotions, and makes optimal suggestions based on the results, which are then shared with beauty salons. In particular, the system provides a system that allows users to apply their preferred hairstyles and colors to their own facial photos, and based on the results of a motion simulation, allows beauty salons to more accurately implement treatments.

[0805] Overall system overview

[0806] This system is broadly composed of the following elements:

[0807] User Device

[0808] server

[0809] Beauty salon terminal

[0810] Emotion Engine

[0811] 1. User Device

[0812] The user terminal provides an interface for users to take or upload their own images and simulate hairstyles and hair colors. Specifically, the following functions are included:

[0813] Image input method: The user takes a photo of their face using the camera function or file upload function.

[0814] Display means: Simulation results are displayed to the user in still image or video format.

[0815] Information transmission means: The generated image and reservation information are sent to the beauty salon.

[0816] Emotion input means: Provides an interface for recognizing the user's emotions.

[0817] 2. Server

[0818] The server receives the images sent by the user and uses AI to apply hairstyles and colors, anonymize the images, analyze emotions, etc. It also forwards the generated images to the hair salon.

[0819] Image processing method: Apply different hairstyles and colors to the user's face photo and perform simulations.

[0820] Image processing method: To protect privacy, the user's face photo is anonymized.

[0821] Video generation means: Converts the simulation results from still images into video format, allowing users to check the motion simulation.

[0822] Information transmission means: The generated image and reservation information are sent to the beauty salon.

[0823] Emotion engine: Analyzes user emotions and records and manages emotional data.

[0824] 3. Beauty salon terminal

[0825] The salon terminal checks the image received from the server and projects it onto an AI mannequin, allowing the stylist to use it as a reference for the treatment.

[0826] Hair salon display means: The received image is displayed and used as a reference for treatment.

[0827] Specific examples

[0828] The program processing will be explained below using specific examples.

[0829] Example 1: Simulation flow

[0830] 1. User: Launches the app and takes or uploads a photo of their face.

[0831] The device sends the user's photo to the server.

[0832] 2. Server: The AI ​​module analyzes the received facial photo and cuts out the facial area.

[0833] 3. User: Select a hairstyle and color from the list provided within the app.

[0834] The terminal transmits the selection information to the server.

[0835] 4. Server: Apply the selected style and color to the user's image and generate the simulation result.

[0836] The generated image is sent back to the device.

[0837] 5. User: Check the simulation results using still images and videos, and make fine adjustments if necessary.

[0838] 6. User: Enter their feelings about the simulation results through the emotion input means.

[0839] 7. Server: The emotion engine analyzes the user's emotional data and, based on the results, re-suggests the most suitable hairstyle and color.

[0840] Example 2: Sharing information with a hair salon

[0841] 1. User: Makes a reservation at a beauty salon based on the simulation results.

[0842] The terminal transmits the reservation information and image data to the server.

[0843] 2. Server: Transfers the received reservation information and image data to the beauty salon terminal.

[0844] 3. Beauty salon terminal: The received image is projected onto the AI ​​mannequin and the stylist checks it.

[0845] 4. Beauty salon terminal: Based on the analysis results of the emotion engine, the terminal makes suggestions that will satisfy the user.

[0846] 5. Hair salon: The stylist performs the treatment based on the image and emotional data received.

[0847] The above is an embodiment of the present invention. The present invention facilitates communication between the user and the stylist, and realizes more accurate hair styling that takes the user's feelings into consideration.

[0848] The processing flow will be explained below.

[0849] Step 1:

[0850] A user launches the app and creates a new account or logs in with an existing account.

[0851] Step 2:

[0852] The user uses the camera function within the app to take a photo of their face or upload an existing photo, and the device sends this image data to the server.

[0853] Step 3:

[0854] The server analyzes the facial photos it receives using an AI module, identifies and cuts out the facial features, and stores the image data in a database after facial recognition processing is complete.

[0855] Step 4:

[0856] The user selects their preferred hairstyle and color from a list of hairstyle and color templates provided within the app, and the device sends the selection information to the server.

[0857] Step 5:

[0858] The server passes the selected hairstyle and color information to the AI ​​module, which applies it to the user's image, and returns the resulting image to the device.

[0859] Step 6:

[0860] The device displays the generated images to the user, or if the user wants to simulate the action, they request that the images be generated in animated format.

[0861] Step 7:

[0862] The server receives the request and generates the simulation results in the form of a video, which is then sent back to the device and displayed for the user to review.

[0863] Step 8:

[0864] The user uses the emotion input means based on the simulation result to input the emotion for the simulation result, and the terminal transmits the emotion data to the server.

[0865] Step 9:

[0866] The server analyzes the received emotion data using the emotion engine. Based on the analysis results, it suggests additional hairstyles and colors to increase the user's satisfaction. The results are then sent back to the device.

[0867] Step 10:

[0868] The device displays additional suggested hairstyles and colors to the user, who then selects a final style based on the suggestions.

[0869] Step 11:

[0870] If the user is satisfied with the simulation results, they make a reservation at the beauty salon. They enter the necessary information into the reservation form, attach the simulation results data, and submit it. The terminal then sends the reservation information and image data to the server.

[0871] Step 12:

[0872] The server receives the reservation information and image data and transfers them to the appropriate salon terminal.

[0873] Step 13:

[0874] The image data received by the salon terminal is projected onto an AI mannequin and displayed for the stylist to check.

[0875] Step 14:

[0876] At the salon, the stylist plans the combination of medications and treatment time based on the user's reservation information, the simulation results, and emotional data. The stylist then actually performs the treatment.

[0877] Step 15:

[0878] Users can check the results of the treatment and share feedback within the app if necessary. The device sends the feedback data to the server.

[0879] Example 2

[0880] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0881] Conventional hairstyle simulation systems were unable to provide optimal suggestions that took the user's emotions into account, resulting in frequent discrepancies between the simulation results and the actual results. Furthermore, there was insufficient sharing of images between the user and the hair salon, making it difficult to provide a highly satisfying treatment. There is a need for a method to share more accurate, emotion-based simulation results with the hair salon while maintaining the user's anonymity.

[0882] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0883] In this invention, the server includes image input means that allows a user to simulate hairstyles and hair colors for their own image, image processing means that generates images of the simulated hairstyles and hair colors, display means that allows the user to check the generated images, information transmission means that shares the generated images with the hair salon when making a reservation at the hair salon, display means that allows the hair salon to display and check the received images, emotion input means that recognizes the user's emotions, and an emotion engine that analyzes the user's emotions and makes optimal suggestions based on the emotion data. This makes it possible to suggest optimal hairstyles and hair colors based on the user's emotions, reduces the gap between the simulation results and the actual treatment results, and enables information to be shared effectively with the hair salon while maintaining anonymity.

[0884] "Image input means" refers to a device or function that allows a user to take or upload their own image and input it into the system.

[0885] "Image processing means" refers to a function for applying and generating simulated hairstyles and hair colors to a user's image.

[0886] A "display means" is a device or function for visually presenting the generated image to a user.

[0887] The "information transmission means" is a communication means for sharing the generated image and reservation information with the beauty salon.

[0888] The "emotion input means" is an interface for recognizing the user's emotions and inputting them into the system.

[0889] The "emotion engine" is a function that analyzes the user's emotional data and makes optimal suggestions based on that data.

[0890] "Anonymization processing" is the process of anonymizing a facial photo to protect the user's privacy.

[0891] The "moving image generating means" is a function for converting the image of the simulated hairstyle and hair color from a still image to a moving image format.

[0892] The "beauty salon display means" is a device or function that displays the received image at the beauty salon and serves as a reference for the treatment.

[0893] This invention relates to a system that allows users to simulate hairstyles and hair colors for their own images, recognizes the user's emotions, and makes optimal suggestions based on the results, which are then shared with a hair salon. This system allows users to apply their favorite hairstyles and colors to their own facial photos and check the results as a motion simulation. Furthermore, by sharing the simulation results with the hair salon, the treatment can be carried out more accurately.

[0894] Overall system overview

[0895] This system mainly consists of the following elements:

[0896] User Device

[0897] server

[0898] Beauty salon terminal

[0899] Emotion Engine

[0900] User Device

[0901] The user device provides an interface for users to take or upload their own images and simulate hairstyles and hair colors. Specifically, it includes the following functions:

[0902] Image input method: The user takes a photo of their face using the camera function or file upload function.

[0903] Display means: Simulation results are displayed to the user in still image or video format.

[0904] Information transmission means: The generated image and reservation information are sent to the beauty salon.

[0905] Emotion input means: Provides an interface for recognizing the user's emotions.

[0906] server

[0907] The server receives the images sent by the user and uses AI to apply hairstyles and colors, anonymize the images, analyze emotions, etc. It also forwards the generated images to the hair salon.

[0908] Image processing method: Apply different hairstyles and colors to the user's face photo and perform simulations.

[0909] Image processing method: To protect privacy, the user's face photo is anonymized.

[0910] Video generation means: Converts the simulation results from still images into video format, allowing users to check the motion simulation.

[0911] Information transmission means: The generated image and reservation information are sent to the beauty salon.

[0912] Emotion engine: Analyzes user emotions and records and manages emotional data.

[0913] Beauty salon terminal

[0914] The salon terminal checks the image received from the server and projects it onto an AI mannequin, allowing the stylist to use it as a reference for the treatment.

[0915] Hair salon display means: The received image is displayed and used as a reference for treatment.

[0916] Specific examples

[0917] The processing of the system will be explained in detail below using specific examples.

[0918] Example 1: Simulation flow

[0919] The user launches the app and takes a photo of their face using their smartphone's camera. This image is then sent from the user's device to the server. The server then uses an AI module to analyze the received photo and extract the facial features. This process is performed using Python and OpenCV. The user selects a hairstyle and color, such as a bob cut and red, from a list within the app, and sends this information from the user's device to the server.

[0920] The server applies the received style and color to the user's image and generates a simulation result. TensorFlow and Keras are used for this. The generated simulation results are generated in the form of still images or videos and sent from the server to the user's device. The user can check the simulation results on the app and make fine adjustments as needed.

[0921] Example 2: Sharing information with a hair salon

[0922] The user makes a reservation at the hair salon based on the final simulation results. This reservation information and simulation image are sent from the user's device to the server. The server then forwards the received reservation information and image data to the hair salon's device. The received simulation results are projected onto the AI ​​mannequin at the hair salon's device, where the stylist can check them.

[0923] Prompt Sentence Examples

[0924] "Use the hairstyle simulation app to apply different styles and colors to your face photo and see the simulation results. Then, tell us how you feel about the results."

[0925] "Book a salon appointment based on the hairstyle simulation generated by the app and share the results with your salon."

[0926] This system facilitates communication between the user and the stylist, enabling more accurate hair styling that takes the user's emotions into account.

[0927] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0928] Step 1: Image Input

[0929] The user launches the app. The user takes or selects a photo of their face using the device's camera function or file upload function. The input is the user's photo of their face, and the output is image data stored on the device. Specifically, the smartphone's camera app launches, and the user takes a photo of their face.

[0930] Step 2: Send image

[0931] The device sends the acquired facial photo to the server. The input is image data on the device, and the output is image data transferred to the server. Specifically, the image data is uploaded to the server via the Internet.

[0932] Step 3: Face Recognition and Segmentation

[0933] The AI ​​module analyzes the image received by the server and extracts the facial area. The input is a face photo data, and the output is image data of the extracted facial area. Specifically, it uses Python's OpenCV library to execute a face detection algorithm and identify the facial area.

[0934] Step 4: Choose your hairstyle and color

[0935] The user selects a hairstyle and hair color from options within the app. The input is the user's selection, and the output is the selected style and color information. Specifically, the user taps to select the desired style and color from the thumbnails on the app screen.

[0936] Step 5: Send your selection

[0937] The terminal sends the selected style and color information to the server. The input is the selection information, and the output is the style and color information transferred to the server. Specifically, the selection data is sent to the server via the Internet.

[0938] Step 6: Run the simulation

[0939] The server applies the selected style and color to the user's image and generates a simulation result. The input is the image data of the extracted face area and style and color information, and the output is the generated simulation result image or video data. Specifically, the style and color application process is performed using TensorFlow and Keras.

[0940] Step 7: Send simulation results

[0941] The server returns the generated simulation results to the terminal. The input is the simulation result data, and the output is the simulation result transferred to the terminal. Specifically, the simulation results are sent to the user terminal via the Internet.

[0942] Step 8: Check the simulation results and make adjustments

[0943] The user checks the simulation results and makes fine adjustments as necessary. The input is image or video data of the simulation results, and the output is the final simulation result after adjustments. Specifically, the user views the simulation results on the app screen and makes fine adjustments to the style and color.

[0944] Step 9: Enter emotions

[0945] The user inputs their feelings about the simulation results. The input is emotional information, and the output is emotional data recorded on the device. Specifically, the user selects buttons provided on the app, such as "Satisfied," "Neutral," or "Dissatisfied."

[0946] Step 10: Sending Emotion Data

[0947] The device sends emotional data to the server. The input is the emotional data on the device, and the output is the emotional data transferred to the server. Specifically, the emotional information is sent to the server via the Internet.

[0948] Step 11: Sentiment analysis and result suggestions

[0949] The server's emotion engine analyzes the emotion data and, based on the results, makes new recommendations for optimal hairstyles and colors. The input is emotion data, and the output is recommendation information based on the analysis results. Specifically, emotion analysis is performed using the IBM Watson Emotion Analysis API to generate optimal recommendations.

[0950] Step 12: Make a reservation at the salon and submit the information

[0951] The user makes a reservation at a hair salon based on the final simulation results. The input is the reservation information and the final simulation results, and the output is the reservation information and image data sent to the server. Specifically, the user selects the desired date and time and hair salon on the reservation page within the app, and that information is sent to the server.

[0952] Step 13: Transfer to salon device

[0953] The server transfers the received reservation information and image data to the salon terminal. The input is the reservation information and image data, and the output is the information sent to the salon terminal. Specifically, the data is transferred to the salon terminal using a backend service based on Java Spring Boot.

[0954] Step 14: Projecting onto the AI ​​mannequin

[0955] The salon terminal receives the simulation results and projects them onto an AI mannequin for the stylist to check. The input is the simulation result data, and the output is the simulated image projected onto the AI ​​mannequin. Specifically, a dedicated display and AI mannequin are used to project the simulation results.

[0956] Step 15: Treatment proposal based on emotion analysis results

[0957] Based on the analysis results of the emotion engine, the salon terminal will propose treatments that will satisfy the user. The input is the emotion analysis results, and the output is treatment proposal information. Specifically, the stylist will provide the user with the optimal treatment plan while referring to the emotion analysis results.

[0958] Step 16: Treatment

[0959] The stylist at the hair salon performs the treatment based on the received image and emotional data. The input is the final simulation image and emotional data, and the output is the hairstyle that will be applied to the user. Specifically, the AI ​​performs the actions of cutting and coloring the user's hair based on the simulation on the mannequin.

[0960] (Application example 2)

[0961] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0962] In conventional hairstyle and hair color simulation systems, it was difficult for users to not only simulate hairstyles but also receive appropriate suggestions based on their emotions. Furthermore, there were no examples of virtual stores where users could easily share simulation results and then immediately purchase or make reservations. This meant that users could not fully confirm the satisfaction and suitability of the product before actually purchasing, potentially resulting in a dissatisfying experience.

[0963] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0964] In this invention, the server includes an image input means that allows a user to simulate hairstyles and hair colors for an image of the user, an image processing means that generates an image of the simulated hairstyle and hair color, a display means that allows the user to check the generated image, an information transmission means that shares the generated image with the hair salon when making a reservation at the hair salon, a display means that allows the hair salon to display and check the received image, an emotion analysis means that analyzes the user's emotions and makes optimal suggestions based on the results, an image regeneration means that regenerates an optimal simulation based on the user's emotion data, and a purchase linkage means that allows the user to purchase items that the user likes based on the try-on simulation results. This allows the user to receive appropriate suggestions that match their emotions based on the simulation results, and further allows them to try on simulated items in a virtual store and directly purchase their favorite items.

[0965] "Image input means" means a device or interface that allows a user to take or upload an image of themselves.

[0966] "Image processing means" refers to algorithms or software for applying and simulating hairstyles and hair colors to photographed or uploaded images.

[0967] "Display means" refers to a device or screen that shows the generated simulation image to the user.

[0968] The "information transmission means" is a communication means for transmitting the generated simulation image to a beauty salon or other service provider.

[0969] An "emotion analysis method" is an algorithm or software that analyzes a user's emotions and makes optimal suggestions based on the results.

[0970] The "image regeneration means" is an algorithm or software for regenerating the simulation based on the results of the emotion analysis.

[0971] The "purchase linking method" is a function that allows users to purchase items they like on the spot based on the results of a try-on simulation.

[0972] "Image processing means" refers to algorithms or software used to process a user's image for anonymity.

[0973] The "video generation means" is an algorithm or software for generating a simulation image in video format so that the user can check the operation simulation.

[0974] A "virtual try-on means" is an interface or software that allows a user to simulate trying on clothes using a photograph of the user's face or whole body.

[0975] To implement this invention, it is necessary to build a system in which the user terminal, server, virtual store system, and beauty salon terminal all function in cooperation with each other. This system is composed of the following elements:

[0976] User Device

[0977] User terminals include smartphones, smart glasses, head-mounted displays, etc. They have the following functions:

[0978] Image input method: An interface that allows a user to take or upload a face or full-body photo, for example by selecting an image from a smartphone camera or photo gallery.

[0979] Display means: A screen for displaying simulation results and proposals to users. Simulation images and videos can be viewed.

[0980] Emotion input means: An interface for analyzing user emotions. Uses an AI engine (e.g., IBM Watson, Microsoft Azure Emotion API).

[0981] server

[0982] The server has the necessary hardware and software to perform the following functions:

[0983] Image processing means: Algorithms (e.g., OpenCV, TensorFlow) for applying hairstyles and hair colors to user-submitted images.

[0984] Image processing methods: Algorithms used to anonymize user images (e.g., image blurring or masking).

[0985] Video generation means: Software for converting simulation results into video format and providing operational simulation.

[0986] Sentiment analysis means: Equipped with a function to analyze emotional data and optimize suggestions (e.g., IBM Watson, Microsoft Azure).

[0987] Image regeneration means: An algorithm that regenerates the simulation based on the results of emotion analysis.

[0988] Purchase linkage: A feature that allows users to purchase items they like on the spot.

[0989] The server analyzes the image data received from the user and performs the following data processing and calculations:

[0990] Image analysis: Identify facial and body features using an AI engine (e.g., TensorFlow).

[0991] Apply Hairstyle: Apply the selected hairstyle and hair color to the image.

[0992] Sentiment Analysis: Analyzes emotional data using a sentiment analysis engine to generate optimal suggestions.

[0993] Simulation Regeneration: Regenerate the simulation based on the analysis results.

[0994] Display of simulation results: Generated images and videos are sent back to the user's device.

[0995] Virtual store system

[0996] The virtual store system implements the following functions:

[0997] Virtual try-on tool: An interface for simulating trying on clothes using a photo of the user's face or full body.

[0998] Purchase linking method: Provide users with a link to purchase items they like based on the simulation results.

[0999] Beauty salon terminal

[1000] The salon terminal includes computers and tablets used by salons. It has the following functions:

[1001] Display means: Simulation images and videos received from the server are displayed and used as a reference for treatment.

[1002] Information transmission method: A function for checking reservation information and simulation results and reflecting them in treatment.

[1003] Specific examples

[1004] The processing flow will be explained below with specific examples.

[1005] 1. Image acquisition: Take a photo of your face with your smartphone and upload it to the app.

[1006] 2. Image analysis: Images sent to the server are analyzed using an AI engine (TensorFlow).

[1007] 3. Simulation Generation: Apply the selected hairstyle and color to the image and generate the result.

[1008] 4. Sentiment analysis: The user inputs their feelings about the generated image, which is then analyzed by the sentiment analysis engine.

[1009] 5. Regeneration: Based on the results of sentiment analysis, the optimal simulation is regenerated.

[1010] 6. Result display: Display the simulation results to the user and readjust if necessary.

[1011] 7. Purchasing integration: Try on items in a virtual store system and purchase your favorite items on the spot.

[1012] Prompt Sentence Examples

[1013] Example prompts for generative AI models:

[1014] "Please simulate applying items selected from the clothing list below to a face photo taken or uploaded by the user. Please combine the items from the provided list with the user's image and return the results. Furthermore, please analyze the user's emotional data and suggest optimal items based on their emotions. Please generate a final image containing the suggested results."

[1015] In this way, the present invention provides a comprehensive system that allows users to simulate hairstyles and fashions that are more suitable for them, and by adding emotion analysis and virtual try-on functions, it is possible to increase user satisfaction.

[1016] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1017] Step 1:

[1018] Users launch the app using a smartphone, smart glasses, or head-mounted display and take or upload a photo of their face or full body.

[1019] Input: A face or full-body photo of the user.

[1020] Output: Image data captured or uploaded.

[1021] Specific behavior: Launch the camera app and take a photo, or select an image from your photo gallery.

[1022] Step 2:

[1023] The user terminal transmits the captured or uploaded image data to the server.

[1024] Input: Image data captured or uploaded by the user.

[1025] Output: Image data sent to the server.

[1026] What it does: Encodes image data and uploads it to a server via a secure communication protocol.

[1027] Step 3:

[1028] The server analyzes the received image data using an AI engine (e.g., TensorFlow) to identify facial and physical features.

[1029] Input: Image data sent to the server.

[1030] Output: Analyzed facial and body feature data.

[1031] How it works: Image data is fed into the AI ​​model, feature points are extracted, and facial recognition algorithms are applied to identify facial contours and features.

[1032] Step 4:

[1033] The server applies the user's selected hairstyle and color to the image to generate a simulation.

[1034] Input: Analyzed facial and body feature data and user-selected hairstyle and color information.

[1035] Output: Image data of the simulation results.

[1036] What it does: Based on facial feature points, it applies the selected hairstyle and color to generate a realistic simulation image.

[1037] Step 5:

[1038] The server transmits the generated simulation image to the user terminal and displays it.

[1039] Input: Image data of the simulation results.

[1040] Output: The simulated image displayed on the user's device.

[1041] Specific operation: Encode the simulated image and send it to the user's device, where it is decoded and displayed on the screen.

[1042] Step 6:

[1043] The user checks the simulation results and inputs his / her feelings about the results using the emotion input means.

[1044] Input: Simulation result display and user emotion data.

[1045] Output: The input information, including emotion data.

[1046] Specific behavior: Look at the simulation results, make decisions, and operate the interface for emotion input.

[1047] Step 7:

[1048] The server analyzes the emotional data using an emotion analysis engine (e.g., IBM Watson) and generates optimal suggestions.

[1049] Input: User emotion data.

[1050] Output: Data with the best suggestions.

[1051] What it does: Passes input data to the sentiment analysis engine, retrieves the analysis results, and generates appropriate suggestions.

[1052] Step 8:

[1053] The server regenerates the simulation based on the emotion analysis results and transmits it again to the user terminal.

[1054] Input: Sentiment analysis results and original simulated image data.

[1055] Output: Regenerated simulated image data.

[1056] Specific operation: A new simulation is generated using hairstyles and colors adjusted based on the results of emotion analysis and transmitted to the user's device.

[1057] Step 9:

[1058] Users can simulate trying on items using a virtual fitting system and purchase the items they like on the spot.

[1059] Input: Regenerated simulation image and user selection information.

[1060] Output: Information about the completed purchase.

[1061] Specific actions: Operate the try-on simulation in the virtual try-on system and use the interface to complete the purchase process.

[1062] The above is a specific processing flow for carrying out the present invention.

[1063] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1064] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1065] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[1066] [Third embodiment]

[1067] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[1068] 5, the data processing system 310 includes the data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[1069] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1070] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[1071] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1072] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1073] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1074] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1075] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1076] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1077] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1078] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[1079] The present invention relates to a system that allows users to simulate hairstyles and hair colors for their own images and share the results with beauty salons. In particular, the system provides a system that allows users to apply their preferred hairstyles and colors to their own facial photos and perform a simulation based on the results of the simulation, enabling more accurate treatments at beauty salons.

[1080] Overall system overview

[1081] This system is broadly composed of the following elements:

[1082] User Device

[1083] server

[1084] Beauty salon terminal

[1085] 1. User Device

[1086] The user terminal provides an interface for users to take or upload their own images and simulate hairstyles and hair colors. Specifically, the following functions are included:

[1087] Image input method: The user takes a photo of their face using the camera function or file upload function.

[1088] Display means: Simulation results are displayed to the user in still image or video format.

[1089] Information transmission means: The generated image and reservation information are sent to the beauty salon.

[1090] 2. Server

[1091] The server receives the image sent by the user and uses AI to apply hairstyles, colors, anonymize, etc. It also forwards the generated image to the hair salon.

[1092] Image processing method: Apply different hairstyles and colors to the user's face photo and perform simulations.

[1093] Image processing method: To protect privacy, the user's face photo is anonymized.

[1094] Video generation means: Converts the simulation results from still images into video format, allowing users to check the motion simulation.

[1095] Information transmission means: The generated image and reservation information are sent to the beauty salon.

[1096] 3. Beauty salon terminal

[1097] The salon terminal checks the image received from the server and projects it onto an AI mannequin, allowing the stylist to use it as a reference for the treatment.

[1098] Hair salon display means: The received image is displayed and used as a reference for treatment.

[1099] Specific examples

[1100] The program processing will be explained below using specific examples.

[1101] Example 1: Simulation flow

[1102] 1. User: Launches the app and takes or uploads a photo of their face.

[1103] The device sends the user's photo to the server.

[1104] 2. Server: The AI ​​module analyzes the received facial photo and cuts out the facial area.

[1105] 3. User: Select a hairstyle and color from the list provided within the app.

[1106] The terminal transmits the selection information to the server.

[1107] 4. Server: Apply the selected style and color to the user's image and generate the simulation result.

[1108] The generated image is sent back to the device.

[1109] 5. User: Check the simulation results using still images and videos, and make fine adjustments if necessary.

[1110] Example 2: Sharing information with a hair salon

[1111] 1. User: Makes a reservation at a beauty salon based on the simulation results.

[1112] The terminal transmits the reservation information and image data to the server.

[1113] 2. Server: Transfers the received reservation information and image data to the beauty salon terminal.

[1114] 3. Beauty salon terminal: The received image is projected onto the AI ​​mannequin and the stylist checks it.

[1115] 4. Hair salon: The stylist performs the treatment based on the image received.

[1116] The above is an embodiment of the present invention. The present invention facilitates communication between the user and the stylist, and allows for more accurate hair styling.

[1117] The processing flow will be explained below.

[1118] Step 1:

[1119] A user launches the app and creates a new account or logs in with an existing account.

[1120] Step 2:

[1121] The user uses the camera function within the app to take a photo of their face or upload an existing photo, and the device sends this image data to the server.

[1122] Step 3:

[1123] The server analyzes the facial photos it receives using an AI module, identifies and cuts out the facial features, and stores the image data in a database after facial recognition processing is complete.

[1124] Step 4:

[1125] The user selects their preferred hairstyle and color from a list of hairstyle and color templates provided within the app, and the device sends the selection information to the server.

[1126] Step 5:

[1127] The server passes the selected hairstyle and color information to the AI ​​module, which applies it to the user's image, and returns the resulting image to the device.

[1128] Step 6:

[1129] The device displays the generated images to the user, or if the user wants to simulate the action, they request that the images be generated in animated format.

[1130] Step 7:

[1131] The server receives the request and generates the simulation results in the form of a video, which is then sent back to the device and displayed for the user to review.

[1132] Step 8:

[1133] If the user is satisfied with the simulation results, they make a reservation at the beauty salon. They enter the necessary information into the reservation form, attach the simulation results data, and submit it. The terminal then sends the reservation information and image data to the server.

[1134] Step 9:

[1135] The server receives the reservation information and image data and transfers them to the appropriate salon terminal.

[1136] Step 10:

[1137] The image data received by the salon terminal is projected onto an AI mannequin and displayed for the stylist to check.

[1138] Step 11:

[1139] At the salon, the stylist plans the combination of medications and treatment time based on the user's reservation information and the simulation results, and then actually performs the treatment.

[1140] Step 12:

[1141] Users can check the results of the treatment and share feedback within the app if necessary.

[1142] Example 1

[1143] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1144] When users receive treatment at a beauty salon, it is difficult for them to accurately communicate their desired hairstyle and hair color. Furthermore, when users simulate hairstyles and hair colors using photos of their own faces, it is difficult to get an idea from still images alone, and privacy is not protected if the images are not anonymized. Since it is not possible to simulate changes in color due to the seasons or weather, the results may not be as expected.

[1145] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1146] In this invention, the server includes a means for projecting the received image onto the AI ​​mannequin, a face detection means for extracting the user's face, and a video generation means for viewing the simulation results in video format. This allows the user to accurately communicate their desired hairstyle and hair color, and viewing the simulation results in video makes it easier to understand the image, while also protecting privacy through anonymity. Furthermore, by simulating color changes due to seasons and weather, it is possible to achieve the expected results.

[1147] "Image input means" refers to the means used by a user to take or upload a photo of their face.

[1148] The "image processing means" is a means for applying a hairstyle and hair color to a facial photo sent by a user and performing a simulation.

[1149] The "display means" is a means for displaying the generated simulation results so that the user can check them.

[1150] The "information transmission means" is a means for transmitting the generated simulation image to the beauty salon.

[1151] "Means for projecting the received image onto the AI ​​mannequin" refers to the means by which the beauty salon projects the received image onto the AI ​​mannequin and visually confirms it.

[1152] The "face detection means" is a means for detecting and extracting the user's face, and mainly uses a deep learning model.

[1153] The "moving image generating means" is a means for generating simulation results in the form of not only still images but also moving images, allowing the user to check the operation simulation.

[1154] "Image processing means" refers to a means of anonymizing a user's facial photograph to protect their privacy.

[1155] The "data compression means" is a means for compressing the generated image data to improve communication efficiency.

[1156] The "function to simulate color changes due to seasons and weather" is a function for simulating changes in hair color due to environmental conditions.

[1157] This system allows users to simulate hairstyles and hair colors on their own images and share them with hair salons. This allows users to check their desired style in advance and ensure smooth and accurate treatment at the salon. The system consists of three main components: a user terminal, a server, and a salon terminal.

[1158] User Device

[1159] The user device provides an interface that allows users to take or upload their own images and simulate hairstyles and hair colors. Specifically, it has the following functions. First, as an image input means, the camera function or file upload function of a smartphone or tablet is used. The user can launch the app and take or upload a photo of their face. Next, as a display means, the simulation results are displayed to the user in still image or video format. This allows the user to visually check various hairstyles and color variations. Finally, as an information transmission means, the generated simulation image and reservation information can be sent to the hair salon.

[1160] server

[1161] The server receives images sent by users and uses AI to apply hairstyles and colors, and anonymize them. It also has the function of transmitting the generated images to the hair salon. Specifically, the functions are as follows: First, as an image processing method, an AI module (e.g., a deep learning model using TensorFlow or PyTorch) is used to analyze the user's facial photo and cut out the facial area. Next, as an image processing method, anonymization is performed on the user's facial photo to protect privacy. Finally, as a video generation method, the simulation results of still images are converted into video format so that the user can check the movement simulation. Finally, as an information transmission method, the generated images and reservation information are sent to the hair salon.

[1162] Beauty salon terminal

[1163] The salon terminal checks the image received from the server and projects it onto an AI mannequin, allowing the stylist to use it as a reference for the treatment. Specifically, it has the following functions. First, as a salon display means, it displays the received image using a display or projector, allowing the stylist to check the treatment details. Furthermore, by projecting the received image onto the AI ​​mannequin, the style can be visually checked in real time.

[1164] Specific examples

[1165] Example 1: Simulation flow

[1166] 1. User: Launches the app and takes or uploads a photo of their face.

[1167] The device sends the user's photo to the server.

[1168] 2. Server: The received face photo is analyzed using an AI module and facial features are extracted. For example, modules using OpenCV or deep learning are used.

[1169] 3. User: Select a hairstyle and color from the list provided within the app.

[1170] The terminal transmits the selection information to the server.

[1171] 4. Server: Apply the selected style and color to the user's image and generate the simulation result.

[1172] The generated image is sent back to the device.

[1173] 5. User: Check the simulation results using still images and videos, and make fine adjustments if necessary.

[1174] Example prompt for a generative AI model:

[1175] Input image: A face photo uploaded by the user

[1176] Hairstyle: Classic Bob

[1177] Hair Color: Blonde

[1178] Output: Generates and returns a simulated image with the specified hairstyle and hair color.

[1179] Example 2: Sharing information with a hair salon

[1180] 1. User: Makes a reservation at a beauty salon based on the simulation results.

[1181] The terminal transmits the reservation information and image data to the server.

[1182] 2. Server: Transfers the received reservation information and image data to the beauty salon terminal.

[1183] 3. Beauty salon terminal: The received image is projected onto the AI ​​mannequin and the stylist checks it.

[1184] 4. Hair salon: The stylist performs the treatment based on the image received.

[1185] This allows users to efficiently simulate their desired hairstyle and color, facilitating smooth communication with the hairdresser. By using this system, users can check their appearance in advance, resulting in a more satisfying experience at the hairdresser.

[1186] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1187] Step 1:

[1188] User: Launches the app and takes or uploads a photo of their face.

[1189] Specific actions include taking a photo of your face using the camera on your smartphone or tablet, or selecting and uploading an existing image file from a folder.

[1190] Input: User's face photo (image file)

[1191] Output: Facial photo data is saved on the user's device.

[1192] Step 2:

[1193] Terminal: The acquired facial photo is compressed and sent to a server via the Internet.

[1194] Specifically, the image file is encoded in JPEG format and sent to the server via an HTTP POST request.

[1195] Input: Facial photo data (image file)

[1196] Output: Compressed face photo data is sent to the server.

[1197] Step 3:

[1198] Server: The received image is analyzed using an AI module, and facial features are detected and extracted.

[1199] Specifically, it applies a facial recognition algorithm using deep learning libraries such as OpenCV and TensorFlow to identify and cut out the facial area.

[1200] Input: Compressed face photo data

[1201] Output: Image data of the cut-out face area

[1202] Step 4:

[1203] User: Selects a hairstyle and color from a list provided through the in-app interface.

[1204] Specifically, you click on the drop-down menus and icons displayed within the app to select the desired hairstyle and color.

[1205] Input: User's selection information (hair style and color)

[1206] Output: The selected hairstyle and color information is saved on the user's device.

[1207] Step 5:

[1208] Terminal: Sends the selection information to the server.

[1209] Specifically, the selected hairstyle and color information is encoded in JSON format and sent to the server via an HTTP POST request.

[1210] Input: User's selection information (hair style and color)

[1211] Output: The selection is sent to the server.

[1212] Step 6:

[1213] Server: Based on the selection information, apply the simulation to the user's face photo.

[1214] Specifically, it uses generative AI models such as TensorFlow and PyTorch to apply hairstyles and colors to a user's image, generating a realistic simulation.

[1215] Input: Image data of the cut-out face, selection information (hairstyle and color)

[1216] Output: Image data of the simulation results

[1217] Example prompt for a generative AI model:

[1218] Input image: A face photo uploaded by the user

[1219] Hairstyle: Classic Bob

[1220] Hair Color: Blonde

[1221] Output: Generates and returns a simulated image with the specified hairstyle and hair color.

[1222] Step 7:

[1223] Server: Generates simulation results in video format and sends them back to the user's device.

[1224] Specifically, the simulation results of still images are converted into video format (MP4, etc.) and sent to the user terminal as an HTTP response.

[1225] Input: Still image data of simulation results

[1226] Output: The simulation result video data is sent to the user's device.

[1227] Step 8:

[1228] User: Review the returned simulation results and make any necessary adjustments.

[1229] Specifically, you can change the hairstyle or color again within the app, start the simulation again, and confirm the final result.

[1230] Input: Simulation result video data

[1231] Output: Final simulation result data

[1232] Step 9:

[1233] User: Sends the final simulation results and reservation information to the hair salon.

[1234] Specifically, the reservation information (date and time, desired treatment) and a simulation image are sent to the server.

[1235] Input: Final simulation result data, reservation information

[1236] Output: The final simulation results and reservation information are sent to the server.

[1237] Step 10:

[1238] Server: Transfers the received reservation information and image data to the beauty salon terminal.

[1239] Specifically, the reservation information and the simulation image are sent to the beauty salon terminal.

[1240] Input: Final simulation results and reservation information

[1241] Output: Reservation information and image data are sent to the salon terminal.

[1242] Step 11:

[1243] Beauty salon terminal: The received image data is projected onto an AI mannequin for the stylist to check.

[1244] Specifically, the received image is displayed using a display or projector.

[1245] Input: Image data, reservation information

[1246] Output: Treatment is performed based on the projected image data.

[1247] The above are the specific processing steps of the system, which are a series of steps to simulate the hairstyle and hair color desired by the user and reflect them in the treatment at the beauty salon.

[1248] (Application example 1)

[1249] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1250] Conventional hair simulation systems have difficulty in sharing simulation results between users and hair salons, which can lead to misunderstandings during treatment. It is also difficult to check and fine-tune simulation results in real time, making it difficult for users to accurately communicate their desired style.

[1251] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1252] In this invention, the server includes means for receiving a facial photo taken or uploaded by a user terminal and analyzing it with an AI module, means for applying a selected hairstyle and hair color to the user's facial photo and returning the generated image, and an AI module for receiving the user's facial photo and generating a hairstyle and color simulation result in real time, allowing the user to simulate a hairstyle and hair color for their own image, check the result in real time, and share it with a hair salon.

[1253] "User terminal" means a device on which a user takes or uploads an image of themselves and simulates hairstyles and hair colors.

[1254] "Image input means" refers to technology that provides a camera function and a file upload function for users to take a photo of their face.

[1255] The "image processing means" is a technology that applies the selected hairstyle and hair color to a user's facial photograph and generates a simulation result.

[1256] "Display means" refers to technology that allows users to check the generated simulation results in the form of still images or videos.

[1257] "Information transmission means" refers to the technology that transmits the generated image and reservation information to the beauty salon.

[1258] The "hair salon display means" is a technology that allows the hair salon to display the received image and use it as a reference for treatment.

[1259] A "server" is a computer system that receives data sent from a user terminal and returns the results of analysis or simulation.

[1260] The "AI module" is a software component that uses artificial intelligence technology to analyze a user's image and apply hairstyles and hair colors.

[1261] The "interface means" refers to a user interface technology that allows the user to check the simulation results in real time and make fine adjustments as necessary.

[1262] This system allows users to simulate hairstyles and hair colors using a smartphone, smart glasses, or a head-mounted display on a photo of their face, and then share the results with a beauty salon. The specific system configuration and process are described below.

[1263] System Configuration

[1264] This system is broadly composed of a user terminal, a server, and a beauty salon terminal.

[1265] User Device

[1266] Image input means: Provides a camera function and file upload function for users to take a photo of their face.

[1267] Display means: Allows users to check the generated simulation results in still image or video format.

[1268] Information transmission means: The generated image and reservation information are transmitted to the server.

[1269] server

[1270] AI module: Receives images sent from the user's device and applies the selected hairstyle and hair color to the user's face photo.

[1271] Image processing means: Analyzes the user's facial photo and applies different hairstyles and hair colors.

[1272] Information transmission means: The generated image is sent back to the user's terminal and, if necessary, forwarded to the beauty salon.

[1273] Beauty salon terminal

[1274] Hair salon display means: The received image is displayed so that the stylist can use it as a reference for the treatment.

[1275] Hardware and software used

[1276] Hardware: The server is a computer system equipped with a high-performance CPU / GPU, and the user terminals are smartphones, smart glasses, and head-mounted displays.

[1277] Software: Python, FastAPI, OpenCV, deep learning frameworks (e.g. TensorFlow, PyTorch)

[1278] Data processing and calculation

[1279] The server receives the facial photo sent by the user and analyzes it using an AI module. It detects the facial area and applies the selected hairstyle and hair color to that area. The processed simulation results are sent back to the user's device, where the user can check and fine-tune in real time. If necessary, the simulation results can also be shared with the hair salon to help with the treatment.

[1280] Specific examples

[1281] For example, if a user wants to try a particular hairstyle, they open the smartphone app and take a photo of their face. Next, they select the desired style and color (e.g., bob cut + blue), and the app sends the image and selection information to the server. The server's AI module analyzes the image and generates a simulation result applying the selected style and color. The generated result is sent back to the user's device, where the user can review and fine-tune in real time. Once they are satisfied with the result, they can share it directly with the hair salon and complete the reservation.

[1282] Prompt Sentence Examples

[1283] "Apply a bob cut hairstyle and change the color to blue to the following photo of your face. Save the resulting image and send it back to the user."

[1284] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1285] Step 1:

[1286] Launch the app on the user's device and take or upload a photo of their face.

[1287] Input: A user-selected face photo.

[1288] Data processing: The app acquires a facial photo and converts it into a format that can be sent to the server.

[1289] Output: The converted facial photo data is sent to the server.

[1290] Step 2:

[1291] The server receives the facial photo sent from the user's device and analyzes it using an AI module.

[1292] Input: Facial photo data sent from the user device.

[1293] Data calculation: The AI ​​module analyzes the facial photo and detects the facial area.

[1294] Output: Face region detected data.

[1295] Step 3:

[1296] The user selects a hairstyle and color from a list provided within the app.

[1297] Input: User-selected hairstyle and color information.

[1298] Data processing: Send the selected information to the server.

[1299] Output: The selected hairstyle and color information is sent to the server.

[1300] Step 4:

[1301] Based on the facial photo and selection information received by the server, a simulation is performed to apply a hairstyle and color.

[1302] Input: Face region detected data and user selected hairstyle and color information.

[1303] Data calculation: The AI ​​module applies hairstyle and color to the face photo.

[1304] Output: Simulation results are generated.

[1305] Step 5:

[1306] The generated simulation results are returned to the user's terminal and displayed.

[1307] Input: Server-generated simulation results.

[1308] Data processing: Converting the generated simulation results into a displayable format.

[1309] Output: Simulation results sent to the user terminal.

[1310] Step 6:

[1311] The user checks the simulation results and makes fine adjustments as necessary.

[1312] Input: Simulation results sent back to the user.

[1313] What it does: Users can see the results in real time and make any necessary adjustments within the app.

[1314] Output: The final simulation result that you are satisfied with.

[1315] Step 7:

[1316] The user makes a reservation at a beauty salon based on the simulation results.

[1317] Input: Final satisfactory simulation results and reservation information.

[1318] Data processing: Simulation results and reservation information are sent to the server.

[1319] Output: Simulation results and reservation information are sent to the beauty salon terminal.

[1320] Step 8:

[1321] The salon terminal displays the received image for the stylist to check.

[1322] Input: Simulation results and reservation information sent from the server.

[1323] Specific operation: The salon terminal displays an image, which the stylist uses as a reference for the treatment.

[1324] Output: Satisfactory results based on the treatment are achieved.

[1325] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1326] The present invention relates to a system that allows users to simulate hairstyles and hair colors for their own images, recognizes the user's emotions, and makes optimal suggestions based on the results, which are then shared with beauty salons. In particular, the system provides a system that allows users to apply their preferred hairstyles and colors to their own facial photos, and based on the results of a motion simulation, allows beauty salons to more accurately implement treatments.

[1327] Overall system overview

[1328] This system is broadly composed of the following elements:

[1329] User Device

[1330] server

[1331] Beauty salon terminal

[1332] Emotion Engine

[1333] 1. User Device

[1334] The user terminal provides an interface for users to take or upload their own images and simulate hairstyles and hair colors. Specifically, the following functions are included:

[1335] Image input method: The user takes a photo of their face using the camera function or file upload function.

[1336] Display means: Simulation results are displayed to the user in still image or video format.

[1337] Information transmission means: The generated image and reservation information are sent to the beauty salon.

[1338] Emotion input means: Provides an interface for recognizing the user's emotions.

[1339] 2. Server

[1340] The server receives the images sent by the user and uses AI to apply hairstyles and colors, anonymize the images, analyze emotions, etc. It also forwards the generated images to the hair salon.

[1341] Image processing method: Apply different hairstyles and colors to the user's face photo and perform simulations.

[1342] Image processing method: To protect privacy, the user's face photo is anonymized.

[1343] Video generation means: Converts the simulation results from still images into video format, allowing users to check the motion simulation.

[1344] Information transmission means: The generated image and reservation information are sent to the beauty salon.

[1345] Emotion engine: Analyzes user emotions and records and manages emotional data.

[1346] 3. Beauty salon terminal

[1347] The salon terminal checks the image received from the server and projects it onto an AI mannequin, allowing the stylist to use it as a reference for the treatment.

[1348] Hair salon display means: The received image is displayed and used as a reference for treatment.

[1349] Specific examples

[1350] The program processing will be explained below using specific examples.

[1351] Example 1: Simulation flow

[1352] 1. User: Launches the app and takes or uploads a photo of their face.

[1353] The device sends the user's photo to the server.

[1354] 2. Server: The AI ​​module analyzes the received facial photo and cuts out the facial area.

[1355] 3. User: Select a hairstyle and color from the list provided within the app.

[1356] The terminal transmits the selection information to the server.

[1357] 4. Server: Apply the selected style and color to the user's image and generate the simulation result.

[1358] The generated image is sent back to the device.

[1359] 5. User: Check the simulation results using still images and videos, and make fine adjustments if necessary.

[1360] 6. User: Enter their feelings about the simulation results through the emotion input means.

[1361] 7. Server: The emotion engine analyzes the user's emotional data and, based on the results, re-suggests the most suitable hairstyle and color.

[1362] Example 2: Sharing information with a hair salon

[1363] 1. User: Makes a reservation at a beauty salon based on the simulation results.

[1364] The terminal transmits the reservation information and image data to the server.

[1365] 2. Server: Transfers the received reservation information and image data to the beauty salon terminal.

[1366] 3. Beauty salon terminal: The received image is projected onto the AI ​​mannequin and the stylist checks it.

[1367] 4. Beauty salon terminal: Based on the analysis results of the emotion engine, the terminal makes suggestions that will satisfy the user.

[1368] 5. Hair salon: The stylist performs the treatment based on the image and emotional data received.

[1369] The above is an embodiment of the present invention. The present invention facilitates communication between the user and the stylist, and realizes more accurate hair styling that takes the user's feelings into consideration.

[1370] The processing flow will be explained below.

[1371] Step 1:

[1372] A user launches the app and creates a new account or logs in with an existing account.

[1373] Step 2:

[1374] The user uses the camera function within the app to take a photo of their face or upload an existing photo, and the device sends this image data to the server.

[1375] Step 3:

[1376] The server analyzes the facial photos it receives using an AI module, identifies and cuts out the facial features, and stores the image data in a database after facial recognition processing is complete.

[1377] Step 4:

[1378] The user selects their preferred hairstyle and color from a list of hairstyle and color templates provided within the app, and the device sends the selection information to the server.

[1379] Step 5:

[1380] The server passes the selected hairstyle and color information to the AI ​​module, which applies it to the user's image, and returns the resulting image to the device.

[1381] Step 6:

[1382] The device displays the generated images to the user, or if the user wants to simulate the action, they request that the images be generated in animated format.

[1383] Step 7:

[1384] The server receives the request and generates the simulation results in the form of a video, which is then sent back to the device and displayed for the user to review.

[1385] Step 8:

[1386] The user uses the emotion input means based on the simulation result to input the emotion for the simulation result, and the terminal transmits the emotion data to the server.

[1387] Step 9:

[1388] The server analyzes the received emotion data using the emotion engine. Based on the analysis results, it suggests additional hairstyles and colors to increase the user's satisfaction. The results are then sent back to the device.

[1389] Step 10:

[1390] The device displays additional suggested hairstyles and colors to the user, who then selects a final style based on the suggestions.

[1391] Step 11:

[1392] If the user is satisfied with the simulation results, they make a reservation at the beauty salon. They enter the necessary information into the reservation form, attach the simulation results data, and submit it. The terminal then sends the reservation information and image data to the server.

[1393] Step 12:

[1394] The server receives the reservation information and image data and transfers them to the appropriate salon terminal.

[1395] Step 13:

[1396] The image data received by the salon terminal is projected onto an AI mannequin and displayed for the stylist to check.

[1397] Step 14:

[1398] At the salon, the stylist plans the combination of medications and treatment time based on the user's reservation information, the simulation results, and emotional data. The stylist then actually performs the treatment.

[1399] Step 15:

[1400] Users can check the results of the treatment and share feedback within the app if necessary. The device sends the feedback data to the server.

[1401] Example 2

[1402] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1403] Conventional hairstyle simulation systems were unable to provide optimal suggestions that took the user's emotions into account, resulting in frequent discrepancies between the simulation results and the actual results. Furthermore, there was insufficient sharing of images between the user and the hair salon, making it difficult to provide a highly satisfying treatment. There is a need for a method to share more accurate, emotion-based simulation results with the hair salon while maintaining the user's anonymity.

[1404] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1405] In this invention, the server includes image input means that allows a user to simulate hairstyles and hair colors for their own image, image processing means that generates images of the simulated hairstyles and hair colors, display means that allows the user to check the generated images, information transmission means that shares the generated images with the hair salon when making a reservation at the hair salon, display means that allows the hair salon to display and check the received images, emotion input means that recognizes the user's emotions, and an emotion engine that analyzes the user's emotions and makes optimal suggestions based on the emotion data. This makes it possible to suggest optimal hairstyles and hair colors based on the user's emotions, reduces the gap between the simulation results and the actual treatment results, and enables information to be shared effectively with the hair salon while maintaining anonymity.

[1406] "Image input means" refers to a device or function that allows a user to take or upload their own image and input it into the system.

[1407] "Image processing means" refers to a function for applying and generating simulated hairstyles and hair colors to a user's image.

[1408] A "display means" is a device or function for visually presenting the generated image to a user.

[1409] The "information transmission means" is a communication means for sharing the generated image and reservation information with the beauty salon.

[1410] The "emotion input means" is an interface for recognizing the user's emotions and inputting them into the system.

[1411] The "emotion engine" is a function that analyzes the user's emotional data and makes optimal suggestions based on that data.

[1412] "Anonymization processing" is the process of anonymizing a facial photo to protect the user's privacy.

[1413] The "moving image generating means" is a function for converting the image of the simulated hairstyle and hair color from a still image to a moving image format.

[1414] The "beauty salon display means" is a device or function that displays the received image at the beauty salon and serves as a reference for the treatment.

[1415] This invention relates to a system that allows users to simulate hairstyles and hair colors for their own images, recognizes the user's emotions, and makes optimal suggestions based on the results, which are then shared with a hair salon. This system allows users to apply their favorite hairstyles and colors to their own facial photos and check the results as a motion simulation. Furthermore, by sharing the simulation results with the hair salon, the treatment can be carried out more accurately.

[1416] Overall system overview

[1417] This system mainly consists of the following elements:

[1418] User Device

[1419] server

[1420] Beauty salon terminal

[1421] Emotion Engine

[1422] User Device

[1423] The user device provides an interface for users to take or upload their own images and simulate hairstyles and hair colors. Specifically, it includes the following functions:

[1424] Image input method: The user takes a photo of their face using the camera function or file upload function.

[1425] Display means: Simulation results are displayed to the user in still image or video format.

[1426] Information transmission means: The generated image and reservation information are sent to the beauty salon.

[1427] Emotion input means: Provides an interface for recognizing the user's emotions.

[1428] server

[1429] The server receives the images sent by the user and uses AI to apply hairstyles and colors, anonymize the images, analyze emotions, etc. It also forwards the generated images to the hair salon.

[1430] Image processing method: Apply different hairstyles and colors to the user's face photo and perform simulations.

[1431] Image processing method: To protect privacy, the user's face photo is anonymized.

[1432] Video generation means: Converts the simulation results from still images into video format, allowing users to check the motion simulation.

[1433] Information transmission means: The generated image and reservation information are sent to the beauty salon.

[1434] Emotion engine: Analyzes user emotions and records and manages emotional data.

[1435] Beauty salon terminal

[1436] The salon terminal checks the image received from the server and projects it onto an AI mannequin, allowing the stylist to use it as a reference for the treatment.

[1437] Hair salon display means: The received image is displayed and used as a reference for treatment.

[1438] Specific examples

[1439] The processing of the system will be explained in detail below using specific examples.

[1440] Example 1: Simulation flow

[1441] The user launches the app and takes a photo of their face using their smartphone's camera. This image is then sent from the user's device to the server. The server then uses an AI module to analyze the received photo and extract the facial features. This process is performed using Python and OpenCV. The user selects a hairstyle and color, such as a bob cut and red, from a list within the app, and sends this information from the user's device to the server.

[1442] The server applies the received style and color to the user's image and generates a simulation result. TensorFlow and Keras are used for this. The generated simulation results are generated in the form of still images or videos and sent from the server to the user's device. The user can check the simulation results on the app and make fine adjustments as needed.

[1443] Example 2: Sharing information with a hair salon

[1444] The user makes a reservation at the hair salon based on the final simulation results. This reservation information and simulation image are sent from the user's device to the server. The server then forwards the received reservation information and image data to the hair salon's device. The received simulation results are projected onto the AI ​​mannequin at the hair salon's device, where the stylist can check them.

[1445] Prompt Sentence Examples

[1446] "Use the hairstyle simulation app to apply different styles and colors to your face photo and see the simulation results. Then, tell us how you feel about the results."

[1447] "Book a salon appointment based on the hairstyle simulation generated by the app and share the results with your salon."

[1448] This system facilitates communication between the user and the stylist, enabling more accurate hair styling that takes the user's emotions into account.

[1449] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1450] Step 1: Image Input

[1451] The user launches the app. The user takes or selects a photo of their face using the device's camera function or file upload function. The input is the user's photo of their face, and the output is image data stored on the device. Specifically, the smartphone's camera app launches, and the user takes a photo of their face.

[1452] Step 2: Send image

[1453] The device sends the acquired facial photo to the server. The input is image data on the device, and the output is image data transferred to the server. Specifically, the image data is uploaded to the server via the Internet.

[1454] Step 3: Face Recognition and Segmentation

[1455] The AI ​​module analyzes the image received by the server and extracts the facial area. The input is a face photo data, and the output is image data of the extracted facial area. Specifically, it uses Python's OpenCV library to execute a face detection algorithm and identify the facial area.

[1456] Step 4: Choose your hairstyle and color

[1457] The user selects a hairstyle and hair color from options within the app. The input is the user's selection, and the output is the selected style and color information. Specifically, the user taps to select the desired style and color from the thumbnails on the app screen.

[1458] Step 5: Send your selection

[1459] The terminal sends the selected style and color information to the server. The input is the selection information, and the output is the style and color information transferred to the server. Specifically, the selection data is sent to the server via the Internet.

[1460] Step 6: Run the simulation

[1461] The server applies the selected style and color to the user's image and generates a simulation result. The input is the image data of the extracted face area and style and color information, and the output is the generated simulation result image or video data. Specifically, the style and color application process is performed using TensorFlow and Keras.

[1462] Step 7: Send simulation results

[1463] The server returns the generated simulation results to the terminal. The input is the simulation result data, and the output is the simulation result transferred to the terminal. Specifically, the simulation results are sent to the user terminal via the Internet.

[1464] Step 8: Check the simulation results and make adjustments

[1465] The user checks the simulation results and makes fine adjustments as necessary. The input is image or video data of the simulation results, and the output is the final simulation result after adjustments. Specifically, the user views the simulation results on the app screen and makes fine adjustments to the style and color.

[1466] Step 9: Enter emotions

[1467] The user inputs their feelings about the simulation results. The input is emotional information, and the output is emotional data recorded on the device. Specifically, the user selects buttons provided on the app, such as "Satisfied," "Neutral," or "Dissatisfied."

[1468] Step 10: Sending Emotion Data

[1469] The device sends emotional data to the server. The input is the emotional data on the device, and the output is the emotional data transferred to the server. Specifically, the emotional information is sent to the server via the Internet.

[1470] Step 11: Sentiment analysis and result suggestions

[1471] The server's emotion engine analyzes the emotion data and, based on the results, makes new recommendations for optimal hairstyles and colors. The input is emotion data, and the output is recommendation information based on the analysis results. Specifically, emotion analysis is performed using the IBM Watson Emotion Analysis API to generate optimal recommendations.

[1472] Step 12: Make a reservation at the salon and submit the information

[1473] The user makes a reservation at a hair salon based on the final simulation results. The input is the reservation information and the final simulation results, and the output is the reservation information and image data sent to the server. Specifically, the user selects the desired date and time and hair salon on the reservation page within the app, and that information is sent to the server.

[1474] Step 13: Transfer to salon device

[1475] The server transfers the received reservation information and image data to the salon terminal. The input is the reservation information and image data, and the output is the information sent to the salon terminal. Specifically, the data is transferred to the salon terminal using a backend service based on Java Spring Boot.

[1476] Step 14: Projecting onto the AI ​​mannequin

[1477] The salon terminal receives the simulation results and projects them onto an AI mannequin for the stylist to check. The input is the simulation result data, and the output is the simulated image projected onto the AI ​​mannequin. Specifically, a dedicated display and AI mannequin are used to project the simulation results.

[1478] Step 15: Treatment proposal based on emotion analysis results

[1479] Based on the analysis results of the emotion engine, the salon terminal will propose treatments that will satisfy the user. The input is the emotion analysis results, and the output is treatment proposal information. Specifically, the stylist will provide the user with the optimal treatment plan while referring to the emotion analysis results.

[1480] Step 16: Treatment

[1481] The stylist at the hair salon performs the treatment based on the received image and emotional data. The input is the final simulation image and emotional data, and the output is the hairstyle that will be applied to the user. Specifically, the AI ​​performs the actions of cutting and coloring the user's hair based on the simulation on the mannequin.

[1482] (Application example 2)

[1483] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1484] In conventional hairstyle and hair color simulation systems, it was difficult for users to not only simulate hairstyles but also receive appropriate suggestions based on their emotions. Furthermore, there were no examples of virtual stores where users could easily share simulation results and then immediately purchase or make reservations. This meant that users could not fully confirm the satisfaction and suitability of the product before actually purchasing, potentially resulting in a dissatisfying experience.

[1485] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1486] In this invention, the server includes an image input means that allows a user to simulate hairstyles and hair colors for an image of the user, an image processing means that generates an image of the simulated hairstyle and hair color, a display means that allows the user to check the generated image, an information transmission means that shares the generated image with the hair salon when making a reservation at the hair salon, a display means that allows the hair salon to display and check the received image, an emotion analysis means that analyzes the user's emotions and makes optimal suggestions based on the results, an image regeneration means that regenerates an optimal simulation based on the user's emotion data, and a purchase linkage means that allows the user to purchase items that the user likes based on the try-on simulation results. This allows the user to receive appropriate suggestions that match their emotions based on the simulation results, and further allows them to try on simulated items in a virtual store and directly purchase their favorite items.

[1487] "Image input means" means a device or interface that allows a user to take or upload an image of themselves.

[1488] "Image processing means" refers to algorithms or software for applying and simulating hairstyles and hair colors to photographed or uploaded images.

[1489] "Display means" refers to a device or screen that shows the generated simulation image to the user.

[1490] The "information transmission means" is a communication means for transmitting the generated simulation image to a beauty salon or other service provider.

[1491] An "emotion analysis method" is an algorithm or software that analyzes a user's emotions and makes optimal suggestions based on the results.

[1492] The "image regeneration means" is an algorithm or software for regenerating the simulation based on the results of the emotion analysis.

[1493] The "purchase linking method" is a function that allows users to purchase items they like on the spot based on the results of a try-on simulation.

[1494] "Image processing means" refers to algorithms or software used to process a user's image for anonymity.

[1495] The "video generation means" is an algorithm or software for generating a simulation image in video format so that the user can check the operation simulation.

[1496] A "virtual try-on means" is an interface or software that allows a user to simulate trying on clothes using a photograph of the user's face or whole body.

[1497] To implement this invention, it is necessary to build a system in which the user terminal, server, virtual store system, and beauty salon terminal all function in cooperation with each other. This system is composed of the following elements:

[1498] User Device

[1499] User terminals include smartphones, smart glasses, head-mounted displays, etc. They have the following functions:

[1500] Image input method: An interface that allows a user to take or upload a face or full-body photo, for example by selecting an image from a smartphone camera or photo gallery.

[1501] Display means: A screen for displaying simulation results and proposals to users. Simulation images and videos can be viewed.

[1502] Emotion input means: An interface for analyzing user emotions. Uses an AI engine (e.g., IBM Watson, Microsoft Azure Emotion API).

[1503] server

[1504] The server has the necessary hardware and software to perform the following functions:

[1505] Image processing means: Algorithms (e.g., OpenCV, TensorFlow) for applying hairstyles and hair colors to user-submitted images.

[1506] Image processing methods: Algorithms used to anonymize user images (e.g., image blurring or masking).

[1507] Video generation means: Software for converting simulation results into video format and providing operational simulation.

[1508] Sentiment analysis means: Equipped with a function to analyze emotional data and optimize suggestions (e.g., IBM Watson, Microsoft Azure).

[1509] Image regeneration means: An algorithm that regenerates the simulation based on the results of emotion analysis.

[1510] Purchase linkage: A feature that allows users to purchase items they like on the spot.

[1511] The server analyzes the image data received from the user and performs the following data processing and calculations:

[1512] Image analysis: Identify facial and body features using an AI engine (e.g., TensorFlow).

[1513] Apply Hairstyle: Apply the selected hairstyle and hair color to the image.

[1514] Sentiment Analysis: Analyzes emotional data using a sentiment analysis engine to generate optimal suggestions.

[1515] Simulation Regeneration: Regenerate the simulation based on the analysis results.

[1516] Display of simulation results: Generated images and videos are sent back to the user's device.

[1517] Virtual store system

[1518] The virtual store system implements the following functions:

[1519] Virtual try-on tool: An interface for simulating trying on clothes using a photo of the user's face or full body.

[1520] Purchase linking method: Provide users with a link to purchase items they like based on the simulation results.

[1521] Beauty salon terminal

[1522] The salon terminal includes computers and tablets used by salons. It has the following functions:

[1523] Display means: Simulation images and videos received from the server are displayed and used as a reference for treatment.

[1524] Information transmission method: A function for checking reservation information and simulation results and reflecting them in treatment.

[1525] Specific examples

[1526] The processing flow will be explained below with specific examples.

[1527] 1. Image acquisition: Take a photo of your face with your smartphone and upload it to the app.

[1528] 2. Image analysis: Images sent to the server are analyzed using an AI engine (TensorFlow).

[1529] 3. Simulation Generation: Apply the selected hairstyle and color to the image and generate the result.

[1530] 4. Sentiment analysis: The user inputs their feelings about the generated image, which is then analyzed by the sentiment analysis engine.

[1531] 5. Regeneration: Based on the results of sentiment analysis, the optimal simulation is regenerated.

[1532] 6. Result display: Display the simulation results to the user and readjust if necessary.

[1533] 7. Purchasing integration: Try on items in a virtual store system and purchase your favorite items on the spot.

[1534] Prompt Sentence Examples

[1535] Example prompts for generative AI models:

[1536] "Please simulate applying items selected from the clothing list below to a face photo taken or uploaded by the user. Please combine the items from the provided list with the user's image and return the results. Furthermore, please analyze the user's emotional data and suggest optimal items based on their emotions. Please generate a final image containing the suggested results."

[1537] In this way, the present invention provides a comprehensive system that allows users to simulate hairstyles and fashions that are more suitable for them, and by adding emotion analysis and virtual try-on functions, it is possible to increase user satisfaction.

[1538] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1539] Step 1:

[1540] Users launch the app using a smartphone, smart glasses, or head-mounted display and take or upload a photo of their face or full body.

[1541] Input: A face or full-body photo of the user.

[1542] Output: Image data captured or uploaded.

[1543] Specific behavior: Launch the camera app and take a photo, or select an image from your photo gallery.

[1544] Step 2:

[1545] The user terminal transmits the captured or uploaded image data to the server.

[1546] Input: Image data captured or uploaded by the user.

[1547] Output: Image data sent to the server.

[1548] What it does: Encodes image data and uploads it to a server via a secure communication protocol.

[1549] Step 3:

[1550] The server analyzes the received image data using an AI engine (e.g., TensorFlow) to identify facial and physical features.

[1551] Input: Image data sent to the server.

[1552] Output: Analyzed facial and body feature data.

[1553] How it works: Image data is fed into the AI ​​model, feature points are extracted, and facial recognition algorithms are applied to identify facial contours and features.

[1554] Step 4:

[1555] The server applies the user's selected hairstyle and color to the image to generate a simulation.

[1556] Input: Analyzed facial and body feature data and user-selected hairstyle and color information.

[1557] Output: Image data of the simulation results.

[1558] What it does: Based on facial feature points, it applies the selected hairstyle and color to generate a realistic simulation image.

[1559] Step 5:

[1560] The server transmits the generated simulation image to the user terminal and displays it.

[1561] Input: Image data of the simulation results.

[1562] Output: The simulated image displayed on the user's device.

[1563] Specific operation: Encode the simulated image and send it to the user's device, where it is decoded and displayed on the screen.

[1564] Step 6:

[1565] The user checks the simulation results and inputs his / her feelings about the results using the emotion input means.

[1566] Input: Simulation result display and user emotion data.

[1567] Output: The input information, including emotion data.

[1568] Specific behavior: Look at the simulation results, make decisions, and operate the interface for emotion input.

[1569] Step 7:

[1570] The server analyzes the emotional data using an emotion analysis engine (e.g., IBM Watson) and generates optimal suggestions.

[1571] Input: User emotion data.

[1572] Output: Data with the best suggestions.

[1573] What it does: Passes input data to the sentiment analysis engine, retrieves the analysis results, and generates appropriate suggestions.

[1574] Step 8:

[1575] The server regenerates the simulation based on the emotion analysis results and transmits it again to the user terminal.

[1576] Input: Sentiment analysis results and original simulated image data.

[1577] Output: Regenerated simulated image data.

[1578] Specific operation: A new simulation is generated using hairstyles and colors adjusted based on the results of emotion analysis and transmitted to the user's device.

[1579] Step 9:

[1580] Users can simulate trying on items using a virtual fitting system and purchase the items they like on the spot.

[1581] Input: Regenerated simulation image and user selection information.

[1582] Output: Information about the completed purchase.

[1583] Specific actions: Operate the try-on simulation in the virtual try-on system and use the interface to complete the purchase process.

[1584] The above is a specific processing flow for carrying out the present invention.

[1585] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1586] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1587] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[1588] [Fourth embodiment]

[1589] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[1590] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1591] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1592] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[1593] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1594] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1595] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1596] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[1597] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1598] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1599] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1600] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1601] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1602] The present invention relates to a system that allows users to simulate hairstyles and hair colors for their own images and share the results with beauty salons. In particular, the system provides a system that allows users to apply their preferred hairstyles and colors to their own facial photos and perform a simulation based on the results of the simulation, enabling more accurate treatments at beauty salons.

[1603] Overall system overview

[1604] This system is broadly composed of the following elements:

[1605] User Device

[1606] server

[1607] Beauty salon terminal

[1608] 1. User Device

[1609] The user terminal provides an interface for users to take or upload their own images and simulate hairstyles and hair colors. Specifically, the following functions are included:

[1610] Image input method: The user takes a photo of their face using the camera function or file upload function.

[1611] Display means: Simulation results are displayed to the user in still image or video format.

[1612] Information transmission means: The generated image and reservation information are sent to the beauty salon.

[1613] 2. Server

[1614] The server receives the image sent by the user and uses AI to apply hairstyles, colors, anonymize, etc. It also forwards the generated image to the hair salon.

[1615] Image processing method: Apply different hairstyles and colors to the user's face photo and perform simulations.

[1616] Image processing method: To protect privacy, the user's face photo is anonymized.

[1617] Video generation means: Converts the simulation results from still images into video format, allowing users to check the motion simulation.

[1618] Information transmission means: The generated image and reservation information are sent to the beauty salon.

[1619] 3. Beauty salon terminal

[1620] The salon terminal checks the image received from the server and projects it onto an AI mannequin, allowing the stylist to use it as a reference for the treatment.

[1621] Hair salon display means: The received image is displayed and used as a reference for treatment.

[1622] Specific examples

[1623] The program processing will be explained below using specific examples.

[1624] Example 1: Simulation flow

[1625] 1. User: Launches the app and takes or uploads a photo of their face.

[1626] The device sends the user's photo to the server.

[1627] 2. Server: The AI ​​module analyzes the received facial photo and cuts out the facial area.

[1628] 3. User: Select a hairstyle and color from the list provided within the app.

[1629] The terminal transmits the selection information to the server.

[1630] 4. Server: Apply the selected style and color to the user's image and generate the simulation result.

[1631] The generated image is sent back to the device.

[1632] 5. User: Check the simulation results using still images and videos, and make fine adjustments if necessary.

[1633] Example 2: Sharing information with a hair salon

[1634] 1. User: Makes a reservation at a beauty salon based on the simulation results.

[1635] The terminal transmits the reservation information and image data to the server.

[1636] 2. Server: Transfers the received reservation information and image data to the beauty salon terminal.

[1637] 3. Beauty salon terminal: The received image is projected onto the AI ​​mannequin and the stylist checks it.

[1638] 4. Hair salon: The stylist performs the treatment based on the image received.

[1639] The above is an embodiment of the present invention. The present invention facilitates communication between the user and the stylist, and allows for more accurate hair styling.

[1640] The processing flow will be explained below.

[1641] Step 1:

[1642] A user launches the app and creates a new account or logs in with an existing account.

[1643] Step 2:

[1644] The user uses the camera function within the app to take a photo of their face or upload an existing photo, and the device sends this image data to the server.

[1645] Step 3:

[1646] The server analyzes the facial photos it receives using an AI module, identifies and cuts out the facial features, and stores the image data in a database after facial recognition processing is complete.

[1647] Step 4:

[1648] The user selects their preferred hairstyle and color from a list of hairstyle and color templates provided within the app, and the device sends the selection information to the server.

[1649] Step 5:

[1650] The server passes the selected hairstyle and color information to the AI ​​module, which applies it to the user's image, and returns the resulting image to the device.

[1651] Step 6:

[1652] The device displays the generated images to the user, or if the user wants to simulate the action, they request that the images be generated in animated format.

[1653] Step 7:

[1654] The server receives the request and generates the simulation results in the form of a video, which is then sent back to the device and displayed for the user to review.

[1655] Step 8:

[1656] If the user is satisfied with the simulation results, they make a reservation at the beauty salon. They enter the necessary information into the reservation form, attach the simulation results data, and submit it. The terminal then sends the reservation information and image data to the server.

[1657] Step 9:

[1658] The server receives the reservation information and image data and transfers them to the appropriate salon terminal.

[1659] Step 10:

[1660] The image data received by the salon terminal is projected onto an AI mannequin and displayed for the stylist to check.

[1661] Step 11:

[1662] At the salon, the stylist plans the combination of medications and treatment time based on the user's reservation information and the simulation results, and then actually performs the treatment.

[1663] Step 12:

[1664] Users can check the results of the treatment and share feedback within the app if necessary.

[1665] Example 1

[1666] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1667] When users receive treatment at a beauty salon, it is difficult for them to accurately communicate their desired hairstyle and hair color. Furthermore, when users simulate hairstyles and hair colors using photos of their own faces, it is difficult to get an idea from still images alone, and privacy is not protected if the images are not anonymized. Since it is not possible to simulate changes in color due to the seasons or weather, the results may not be as expected.

[1668] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1669] In this invention, the server includes a means for projecting the received image onto the AI ​​mannequin, a face detection means for extracting the user's face, and a video generation means for viewing the simulation results in video format. This allows the user to accurately communicate their desired hairstyle and hair color, and viewing the simulation results in video makes it easier to understand the image, while also protecting privacy through anonymity. Furthermore, by simulating color changes due to seasons and weather, it is possible to achieve the expected results.

[1670] "Image input means" refers to the means used by a user to take or upload a photo of their face.

[1671] The "image processing means" is a means for applying a hairstyle and hair color to a facial photo sent by a user and performing a simulation.

[1672] The "display means" is a means for displaying the generated simulation results so that the user can check them.

[1673] The "information transmission means" is a means for transmitting the generated simulation image to the beauty salon.

[1674] "Means for projecting the received image onto the AI ​​mannequin" refers to the means by which the beauty salon projects the received image onto the AI ​​mannequin and visually confirms it.

[1675] The "face detection means" is a means for detecting and extracting the user's face, and mainly uses a deep learning model.

[1676] The "moving image generating means" is a means for generating simulation results in the form of not only still images but also moving images, allowing the user to check the operation simulation.

[1677] "Image processing means" refers to a means of anonymizing a user's facial photograph to protect their privacy.

[1678] The "data compression means" is a means for compressing the generated image data to improve communication efficiency.

[1679] The "function to simulate color changes due to seasons and weather" is a function for simulating changes in hair color due to environmental conditions.

[1680] This system allows users to simulate hairstyles and hair colors on their own images and share them with hair salons. This allows users to check their desired style in advance and ensure smooth and accurate treatment at the salon. The system consists of three main components: a user terminal, a server, and a salon terminal.

[1681] User Device

[1682] The user device provides an interface that allows users to take or upload their own images and simulate hairstyles and hair colors. Specifically, it has the following functions. First, as an image input means, the camera function or file upload function of a smartphone or tablet is used. The user can launch the app and take or upload a photo of their face. Next, as a display means, the simulation results are displayed to the user in still image or video format. This allows the user to visually check various hairstyles and color variations. Finally, as an information transmission means, the generated simulation image and reservation information can be sent to the hair salon.

[1683] server

[1684] The server receives images sent by users and uses AI to apply hairstyles and colors, and anonymize them. It also has the function of transmitting the generated images to the hair salon. Specifically, the functions are as follows: First, as an image processing method, an AI module (e.g., a deep learning model using TensorFlow or PyTorch) is used to analyze the user's facial photo and cut out the facial area. Next, as an image processing method, anonymization is performed on the user's facial photo to protect privacy. Finally, as a video generation method, the simulation results of still images are converted into video format so that the user can check the movement simulation. Finally, as an information transmission method, the generated images and reservation information are sent to the hair salon.

[1685] Beauty salon terminal

[1686] The salon terminal checks the image received from the server and projects it onto an AI mannequin, allowing the stylist to use it as a reference for the treatment. Specifically, it has the following functions. First, as a salon display means, it displays the received image using a display or projector, allowing the stylist to check the treatment details. Furthermore, by projecting the received image onto the AI ​​mannequin, the style can be visually checked in real time.

[1687] Specific examples

[1688] Example 1: Simulation flow

[1689] 1. User: Launches the app and takes or uploads a photo of their face.

[1690] The device sends the user's photo to the server.

[1691] 2. Server: The received face photo is analyzed using an AI module and facial features are extracted. For example, modules using OpenCV or deep learning are used.

[1692] 3. User: Select a hairstyle and color from the list provided within the app.

[1693] The terminal transmits the selection information to the server.

[1694] 4. Server: Apply the selected style and color to the user's image and generate the simulation result.

[1695] The generated image is sent back to the device.

[1696] 5. User: Check the simulation results using still images and videos, and make fine adjustments if necessary.

[1697] Example prompt for a generative AI model:

[1698] Input image: A face photo uploaded by the user

[1699] Hairstyle: Classic Bob

[1700] Hair Color: Blonde

[1701] Output: Generates and returns a simulated image with the specified hairstyle and hair color.

[1702] Example 2: Sharing information with a hair salon

[1703] 1. User: Makes a reservation at a beauty salon based on the simulation results.

[1704] The terminal transmits the reservation information and image data to the server.

[1705] 2. Server: Transfers the received reservation information and image data to the beauty salon terminal.

[1706] 3. Beauty salon terminal: The received image is projected onto the AI ​​mannequin and the stylist checks it.

[1707] 4. Hair salon: The stylist performs the treatment based on the image received.

[1708] This allows users to efficiently simulate their desired hairstyle and color, facilitating smooth communication with the hairdresser. By using this system, users can check their appearance in advance, resulting in a more satisfying experience at the hairdresser.

[1709] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1710] Step 1:

[1711] User: Launches the app and takes or uploads a photo of their face.

[1712] Specific actions include taking a photo of your face using the camera on your smartphone or tablet, or selecting and uploading an existing image file from a folder.

[1713] Input: User's face photo (image file)

[1714] Output: Facial photo data is saved on the user's device.

[1715] Step 2:

[1716] Terminal: The acquired facial photo is compressed and sent to a server via the Internet.

[1717] Specifically, the image file is encoded in JPEG format and sent to the server via an HTTP POST request.

[1718] Input: Facial photo data (image file)

[1719] Output: Compressed face photo data is sent to the server.

[1720] Step 3:

[1721] Server: The received image is analyzed using an AI module, and facial features are detected and extracted.

[1722] Specifically, it applies a facial recognition algorithm using deep learning libraries such as OpenCV and TensorFlow to identify and cut out the facial area.

[1723] Input: Compressed face photo data

[1724] Output: Image data of the cut-out face area

[1725] Step 4:

[1726] User: Selects a hairstyle and color from a list provided through the in-app interface.

[1727] Specifically, you click on the drop-down menus and icons displayed within the app to select the desired hairstyle and color.

[1728] Input: User's selection information (hair style and color)

[1729] Output: The selected hairstyle and color information is saved on the user's device.

[1730] Step 5:

[1731] Terminal: Sends the selection information to the server.

[1732] Specifically, the selected hairstyle and color information is encoded in JSON format and sent to the server via an HTTP POST request.

[1733] Input: User's selection information (hair style and color)

[1734] Output: The selection is sent to the server.

[1735] Step 6:

[1736] Server: Based on the selection information, apply the simulation to the user's face photo.

[1737] Specifically, it uses generative AI models such as TensorFlow and PyTorch to apply hairstyles and colors to a user's image, generating a realistic simulation.

[1738] Input: Image data of the cut-out face, selection information (hairstyle and color)

[1739] Output: Image data of the simulation results

[1740] Example prompt for a generative AI model:

[1741] Input image: A face photo uploaded by the user

[1742] Hairstyle: Classic Bob

[1743] Hair Color: Blonde

[1744] Output: Generates and returns a simulated image with the specified hairstyle and hair color.

[1745] Step 7:

[1746] Server: Generates simulation results in video format and sends them back to the user's device.

[1747] Specifically, the simulation results of still images are converted into video format (MP4, etc.) and sent to the user terminal as an HTTP response.

[1748] Input: Still image data of simulation results

[1749] Output: The simulation result video data is sent to the user's device.

[1750] Step 8:

[1751] User: Review the returned simulation results and make any necessary adjustments.

[1752] Specifically, you can change the hairstyle or color again within the app, start the simulation again, and confirm the final result.

[1753] Input: Simulation result video data

[1754] Output: Final simulation result data

[1755] Step 9:

[1756] User: Sends the final simulation results and reservation information to the hair salon.

[1757] Specifically, the reservation information (date and time, desired treatment) and a simulation image are sent to the server.

[1758] Input: Final simulation result data, reservation information

[1759] Output: The final simulation results and reservation information are sent to the server.

[1760] Step 10:

[1761] Server: Transfers the received reservation information and image data to the beauty salon terminal.

[1762] Specifically, the reservation information and the simulation image are sent to the beauty salon terminal.

[1763] Input: Final simulation results and reservation information

[1764] Output: Reservation information and image data are sent to the salon terminal.

[1765] Step 11:

[1766] Beauty salon terminal: The received image data is projected onto an AI mannequin for the stylist to check.

[1767] Specifically, the received image is displayed using a display or projector.

[1768] Input: Image data, reservation information

[1769] Output: Treatment is performed based on the projected image data.

[1770] The above are the specific processing steps of the system, which are a series of steps to simulate the hairstyle and hair color desired by the user and reflect them in the treatment at the beauty salon.

[1771] (Application example 1)

[1772] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1773] Conventional hair simulation systems have difficulty in sharing simulation results between users and hair salons, which can lead to misunderstandings during treatment. It is also difficult to check and fine-tune simulation results in real time, making it difficult for users to accurately communicate their desired style.

[1774] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1775] In this invention, the server includes means for receiving a facial photo taken or uploaded by a user terminal and analyzing it with an AI module, means for applying a selected hairstyle and hair color to the user's facial photo and returning the generated image, and an AI module for receiving the user's facial photo and generating a hairstyle and color simulation result in real time, allowing the user to simulate a hairstyle and hair color for their own image, check the result in real time, and share it with a hair salon.

[1776] "User terminal" means a device on which a user takes or uploads an image of themselves and simulates hairstyles and hair colors.

[1777] "Image input means" refers to technology that provides a camera function and a file upload function for users to take a photo of their face.

[1778] The "image processing means" is a technology that applies the selected hairstyle and hair color to a user's facial photograph and generates a simulation result.

[1779] "Display means" refers to technology that allows users to check the generated simulation results in the form of still images or videos.

[1780] "Information transmission means" refers to the technology that transmits the generated image and reservation information to the beauty salon.

[1781] The "hair salon display means" is a technology that allows the hair salon to display the received image and use it as a reference for treatment.

[1782] A "server" is a computer system that receives data sent from a user terminal and returns the results of analysis or simulation.

[1783] The "AI module" is a software component that uses artificial intelligence technology to analyze a user's image and apply hairstyles and hair colors.

[1784] The "interface means" refers to a user interface technology that allows the user to check the simulation results in real time and make fine adjustments as necessary.

[1785] This system allows users to simulate hairstyles and hair colors using a smartphone, smart glasses, or a head-mounted display on a photo of their face, and then share the results with a beauty salon. The specific system configuration and process are described below.

[1786] System Configuration

[1787] This system is broadly composed of a user terminal, a server, and a beauty salon terminal.

[1788] User Device

[1789] Image input means: Provides a camera function and file upload function for users to take a photo of their face.

[1790] Display means: Allows users to check the generated simulation results in still image or video format.

[1791] Information transmission means: The generated image and reservation information are transmitted to the server.

[1792] server

[1793] AI module: Receives images sent from the user's device and applies the selected hairstyle and hair color to the user's face photo.

[1794] Image processing means: Analyzes the user's facial photo and applies different hairstyles and hair colors.

[1795] Information transmission means: The generated image is sent back to the user's terminal and, if necessary, forwarded to the beauty salon.

[1796] Beauty salon terminal

[1797] Hair salon display means: The received image is displayed so that the stylist can use it as a reference for the treatment.

[1798] Hardware and software used

[1799] Hardware: The server is a computer system equipped with a high-performance CPU / GPU, and the user terminals are smartphones, smart glasses, and head-mounted displays.

[1800] Software: Python, FastAPI, OpenCV, deep learning frameworks (e.g. TensorFlow, PyTorch)

[1801] Data processing and calculation

[1802] The server receives the facial photo sent by the user and analyzes it using an AI module. It detects the facial area and applies the selected hairstyle and hair color to that area. The processed simulation results are sent back to the user's device, where the user can check and fine-tune in real time. If necessary, the simulation results can also be shared with the hair salon to help with the treatment.

[1803] Specific examples

[1804] For example, if a user wants to try a particular hairstyle, they open the smartphone app and take a photo of their face. Next, they select the desired style and color (e.g., bob cut + blue), and the app sends the image and selection information to the server. The server's AI module analyzes the image and generates a simulation result applying the selected style and color. The generated result is sent back to the user's device, where the user can review and fine-tune in real time. Once they are satisfied with the result, they can share it directly with the hair salon and complete the reservation.

[1805] Prompt Sentence Examples

[1806] "Apply a bob cut hairstyle and change the color to blue to the following photo of your face. Save the resulting image and send it back to the user."

[1807] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1808] Step 1:

[1809] Launch the app on the user's device and take or upload a photo of their face.

[1810] Input: A user-selected face photo.

[1811] Data processing: The app acquires a facial photo and converts it into a format that can be sent to the server.

[1812] Output: The converted facial photo data is sent to the server.

[1813] Step 2:

[1814] The server receives the facial photo sent from the user's device and analyzes it using an AI module.

[1815] Input: Facial photo data sent from the user device.

[1816] Data calculation: The AI ​​module analyzes the facial photo and detects the facial area.

[1817] Output: Face region detected data.

[1818] Step 3:

[1819] The user selects a hairstyle and color from a list provided within the app.

[1820] Input: User-selected hairstyle and color information.

[1821] Data processing: Send the selected information to the server.

[1822] Output: The selected hairstyle and color information is sent to the server.

[1823] Step 4:

[1824] Based on the facial photo and selection information received by the server, a simulation is performed to apply a hairstyle and color.

[1825] Input: Face region detected data and user selected hairstyle and color information.

[1826] Data calculation: The AI ​​module applies hairstyle and color to the face photo.

[1827] Output: Simulation results are generated.

[1828] Step 5:

[1829] The generated simulation results are returned to the user's terminal and displayed.

[1830] Input: Server-generated simulation results.

[1831] Data processing: Converting the generated simulation results into a displayable format.

[1832] Output: Simulation results sent to the user terminal.

[1833] Step 6:

[1834] The user checks the simulation results and makes fine adjustments as necessary.

[1835] Input: Simulation results sent back to the user.

[1836] What it does: Users can see the results in real time and make any necessary adjustments within the app.

[1837] Output: The final simulation result that you are satisfied with.

[1838] Step 7:

[1839] The user makes a reservation at a beauty salon based on the simulation results.

[1840] Input: Final satisfactory simulation results and reservation information.

[1841] Data processing: Simulation results and reservation information are sent to the server.

[1842] Output: Simulation results and reservation information are sent to the beauty salon terminal.

[1843] Step 8:

[1844] The salon terminal displays the received image for the stylist to check.

[1845] Input: Simulation results and reservation information sent from the server.

[1846] Specific operation: The salon terminal displays an image, which the stylist uses as a reference for the treatment.

[1847] Output: Satisfactory results based on the treatment are achieved.

[1848] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1849] The present invention relates to a system that allows users to simulate hairstyles and hair colors for their own images, recognizes the user's emotions, and makes optimal suggestions based on the results, which are then shared with beauty salons. In particular, the system provides a system that allows users to apply their preferred hairstyles and colors to their own facial photos, and based on the results of a motion simulation, allows beauty salons to more accurately implement treatments.

[1850] Overall system overview

[1851] This system is broadly composed of the following elements:

[1852] User Device

[1853] server

[1854] Beauty salon terminal

[1855] Emotion Engine

[1856] 1. User Device

[1857] The user terminal provides an interface for users to take or upload their own images and simulate hairstyles and hair colors. Specifically, the following functions are included:

[1858] Image input method: The user takes a photo of their face using the camera function or file upload function.

[1859] Display means: Simulation results are displayed to the user in still image or video format.

[1860] Information transmission means: The generated image and reservation information are sent to the beauty salon.

[1861] Emotion input means: Provides an interface for recognizing the user's emotions.

[1862] 2. Server

[1863] The server receives the images sent by the user and uses AI to apply hairstyles and colors, anonymize the images, analyze emotions, etc. It also forwards the generated images to the hair salon.

[1864] Image processing method: Apply different hairstyles and colors to the user's face photo and perform simulations.

[1865] Image processing method: To protect privacy, the user's face photo is anonymized.

[1866] Video generation means: Converts the simulation results from still images into video format, allowing users to check the motion simulation.

[1867] Information transmission means: The generated image and reservation information are sent to the beauty salon.

[1868] Emotion engine: Analyzes user emotions and records and manages emotional data.

[1869] 3. Beauty salon terminal

[1870] The salon terminal checks the image received from the server and projects it onto an AI mannequin, allowing the stylist to use it as a reference for the treatment.

[1871] Hair salon display means: The received image is displayed and used as a reference for treatment.

[1872] Specific examples

[1873] The program processing will be explained below using specific examples.

[1874] Example 1: Simulation flow

[1875] 1. User: Launches the app and takes or uploads a photo of their face.

[1876] The device sends the user's photo to the server.

[1877] 2. Server: The AI ​​module analyzes the received facial photo and cuts out the facial area.

[1878] 3. User: Select a hairstyle and color from the list provided within the app.

[1879] The terminal transmits the selection information to the server.

[1880] 4. Server: Apply the selected style and color to the user's image and generate the simulation result.

[1881] The generated image is sent back to the device.

[1882] 5. User: Check the simulation results using still images and videos, and make fine adjustments if necessary.

[1883] 6. User: Enter their feelings about the simulation results through the emotion input means.

[1884] 7. Server: The emotion engine analyzes the user's emotional data and, based on the results, re-suggests the most suitable hairstyle and color.

[1885] Example 2: Sharing information with a hair salon

[1886] 1. User: Makes a reservation at a beauty salon based on the simulation results.

[1887] The terminal transmits the reservation information and image data to the server.

[1888] 2. Server: Transfers the received reservation information and image data to the beauty salon terminal.

[1889] 3. Beauty salon terminal: The received image is projected onto the AI ​​mannequin and the stylist checks it.

[1890] 4. Beauty salon terminal: Based on the analysis results of the emotion engine, the terminal makes suggestions that will satisfy the user.

[1891] 5. Hair salon: The stylist performs the treatment based on the image and emotional data received.

[1892] The above is an embodiment of the present invention. The present invention facilitates communication between the user and the stylist, and realizes more accurate hair styling that takes the user's feelings into consideration.

[1893] The processing flow will be explained below.

[1894] Step 1:

[1895] A user launches the app and creates a new account or logs in with an existing account.

[1896] Step 2:

[1897] The user uses the camera function within the app to take a photo of their face or upload an existing photo, and the device sends this image data to the server.

[1898] Step 3:

[1899] The server analyzes the facial photos it receives using an AI module, identifies and cuts out the facial features, and stores the image data in a database after facial recognition processing is complete.

[1900] Step 4:

[1901] The user selects their preferred hairstyle and color from a list of hairstyle and color templates provided within the app, and the device sends the selection information to the server.

[1902] Step 5:

[1903] The server passes the selected hairstyle and color information to the AI ​​module, which applies it to the user's image, and returns the resulting image to the device.

[1904] Step 6:

[1905] The device displays the generated images to the user, or if the user wants to simulate the action, they request that the images be generated in animated format.

[1906] Step 7:

[1907] The server receives the request and generates the simulation results in the form of a video, which is then sent back to the device and displayed for the user to review.

[1908] Step 8:

[1909] The user uses the emotion input means based on the simulation result to input the emotion for the simulation result, and the terminal transmits the emotion data to the server.

[1910] Step 9:

[1911] The server analyzes the received emotion data using the emotion engine. Based on the analysis results, it suggests additional hairstyles and colors to increase the user's satisfaction. The results are then sent back to the device.

[1912] Step 10:

[1913] The device displays additional suggested hairstyles and colors to the user, who then selects a final style based on the suggestions.

[1914] Step 11:

[1915] If the user is satisfied with the simulation results, they make a reservation at the beauty salon. They enter the necessary information into the reservation form, attach the simulation results data, and submit it. The terminal then sends the reservation information and image data to the server.

[1916] Step 12:

[1917] The server receives the reservation information and image data and transfers them to the appropriate salon terminal.

[1918] Step 13:

[1919] The image data received by the salon terminal is projected onto an AI mannequin and displayed for the stylist to check.

[1920] Step 14:

[1921] At the salon, the stylist plans the combination of medications and treatment time based on the user's reservation information, the simulation results, and emotional data. The stylist then actually performs the treatment.

[1922] Step 15:

[1923] Users can check the results of the treatment and share feedback within the app if necessary. The device sends the feedback data to the server.

[1924] Example 2

[1925] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1926] Conventional hairstyle simulation systems were unable to provide optimal suggestions that took the user's emotions into account, resulting in frequent discrepancies between the simulation results and the actual results. Furthermore, there was insufficient sharing of images between the user and the hair salon, making it difficult to provide a highly satisfying treatment. There is a need for a method to share more accurate, emotion-based simulation results with the hair salon while maintaining the user's anonymity.

[1927] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1928] In this invention, the server includes image input means that allows a user to simulate hairstyles and hair colors for their own image, image processing means that generates images of the simulated hairstyles and hair colors, display means that allows the user to check the generated images, information transmission means that shares the generated images with the hair salon when making a reservation at the hair salon, display means that allows the hair salon to display and check the received images, emotion input means that recognizes the user's emotions, and an emotion engine that analyzes the user's emotions and makes optimal suggestions based on the emotion data. This makes it possible to suggest optimal hairstyles and hair colors based on the user's emotions, reduces the gap between the simulation results and the actual treatment results, and enables information to be shared effectively with the hair salon while maintaining anonymity.

[1929] "Image input means" refers to a device or function that allows a user to take or upload their own image and input it into the system.

[1930] "Image processing means" refers to a function for applying and generating simulated hairstyles and hair colors to a user's image.

[1931] A "display means" is a device or function for visually presenting the generated image to a user.

[1932] The "information transmission means" is a communication means for sharing the generated image and reservation information with the beauty salon.

[1933] The "emotion input means" is an interface for recognizing the user's emotions and inputting them into the system.

[1934] The "emotion engine" is a function that analyzes the user's emotional data and makes optimal suggestions based on that data.

[1935] "Anonymization processing" is the process of anonymizing a facial photo to protect the user's privacy.

[1936] The "moving image generating means" is a function for converting the image of the simulated hairstyle and hair color from a still image to a moving image format.

[1937] The "beauty salon display means" is a device or function that displays the received image at the beauty salon and serves as a reference for the treatment.

[1938] This invention relates to a system that allows users to simulate hairstyles and hair colors for their own images, recognizes the user's emotions, and makes optimal suggestions based on the results, which are then shared with a hair salon. This system allows users to apply their favorite hairstyles and colors to their own facial photos and check the results as a motion simulation. Furthermore, by sharing the simulation results with the hair salon, the treatment can be carried out more accurately.

[1939] Overall system overview

[1940] This system mainly consists of the following elements:

[1941] User Device

[1942] server

[1943] Beauty salon terminal

[1944] Emotion Engine

[1945] User Device

[1946] The user device provides an interface for users to take or upload their own images and simulate hairstyles and hair colors. Specifically, it includes the following functions:

[1947] Image input method: The user takes a photo of their face using the camera function or file upload function.

[1948] Display means: Simulation results are displayed to the user in still image or video format.

[1949] Information transmission means: The generated image and reservation information are sent to the beauty salon.

[1950] Emotion input means: Provides an interface for recognizing the user's emotions.

[1951] server

[1952] The server receives the images sent by the user and uses AI to apply hairstyles and colors, anonymize the images, analyze emotions, etc. It also forwards the generated images to the hair salon.

[1953] Image processing method: Apply different hairstyles and colors to the user's face photo and perform simulations.

[1954] Image processing method: To protect privacy, the user's face photo is anonymized.

[1955] Video generation means: Converts the simulation results from still images into video format, allowing users to check the motion simulation.

[1956] Information transmission means: The generated image and reservation information are sent to the beauty salon.

[1957] Emotion engine: Analyzes user emotions and records and manages emotional data.

[1958] Beauty salon terminal

[1959] The salon terminal checks the image received from the server and projects it onto an AI mannequin, allowing the stylist to use it as a reference for the treatment.

[1960] Hair salon display means: The received image is displayed and used as a reference for treatment.

[1961] Specific examples

[1962] The processing of the system will be explained in detail below using specific examples.

[1963] Example 1: Simulation flow

[1964] The user launches the app and takes a photo of their face using their smartphone's camera. This image is then sent from the user's device to the server. The server then uses an AI module to analyze the received photo and extract the facial features. This process is performed using Python and OpenCV. The user selects a hairstyle and color, such as a bob cut and red, from a list within the app, and sends this information from the user's device to the server.

[1965] The server applies the received style and color to the user's image and generates a simulation result. TensorFlow and Keras are used for this. The generated simulation results are generated in the form of still images or videos and sent from the server to the user's device. The user can check the simulation results on the app and make fine adjustments as needed.

[1966] Example 2: Sharing information with a hair salon

[1967] The user makes a reservation at the hair salon based on the final simulation results. This reservation information and simulation image are sent from the user's device to the server. The server then forwards the received reservation information and image data to the hair salon's device. The received simulation results are projected onto the AI ​​mannequin at the hair salon's device, where the stylist can check them.

[1968] Prompt Sentence Examples

[1969] "Use the hairstyle simulation app to apply different styles and colors to your face photo and see the simulation results. Then, tell us how you feel about the results."

[1970] "Book a salon appointment based on the hairstyle simulation generated by the app and share the results with your salon."

[1971] This system facilitates communication between the user and the stylist, enabling more accurate hair styling that takes the user's emotions into account.

[1972] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1973] Step 1: Image Input

[1974] The user launches the app. The user takes or selects a photo of their face using the device's camera function or file upload function. The input is the user's photo of their face, and the output is image data stored on the device. Specifically, the smartphone's camera app launches, and the user takes a photo of their face.

[1975] Step 2: Send image

[1976] The device sends the acquired facial photo to the server. The input is image data on the device, and the output is image data transferred to the server. Specifically, the image data is uploaded to the server via the Internet.

[1977] Step 3: Face Recognition and Segmentation

[1978] The AI ​​module analyzes the image received by the server and extracts the facial area. The input is a face photo data, and the output is image data of the extracted facial area. Specifically, it uses Python's OpenCV library to execute a face detection algorithm and identify the facial area.

[1979] Step 4: Choose your hairstyle and color

[1980] The user selects a hairstyle and hair color from options within the app. The input is the user's selection, and the output is the selected style and color information. Specifically, the user taps to select the desired style and color from the thumbnails on the app screen.

[1981] Step 5: Send your selection

[1982] The terminal sends the selected style and color information to the server. The input is the selection information, and the output is the style and color information transferred to the server. Specifically, the selection data is sent to the server via the Internet.

[1983] Step 6: Run the simulation

[1984] The server applies the selected style and color to the user's image and generates a simulation result. The input is the image data of the extracted face area and style and color information, and the output is the generated simulation result image or video data. Specifically, the style and color application process is performed using TensorFlow and Keras.

[1985] Step 7: Send simulation results

[1986] The server returns the generated simulation results to the terminal. The input is the simulation result data, and the output is the simulation result transferred to the terminal. Specifically, the simulation results are sent to the user terminal via the Internet.

[1987] Step 8: Check the simulation results and make adjustments

[1988] The user checks the simulation results and makes fine adjustments as necessary. The input is image or video data of the simulation results, and the output is the final simulation result after adjustments. Specifically, the user views the simulation results on the app screen and makes fine adjustments to the style and color.

[1989] Step 9: Enter emotions

[1990] The user inputs their feelings about the simulation results. The input is emotional information, and the output is emotional data recorded on the device. Specifically, the user selects buttons provided on the app, such as "Satisfied," "Neutral," or "Dissatisfied."

[1991] Step 10: Sending Emotion Data

[1992] The device sends emotional data to the server. The input is the emotional data on the device, and the output is the emotional data transferred to the server. Specifically, the emotional information is sent to the server via the Internet.

[1993] Step 11: Sentiment analysis and result suggestions

[1994] The server's emotion engine analyzes the emotion data and, based on the results, makes new recommendations for optimal hairstyles and colors. The input is emotion data, and the output is recommendation information based on the analysis results. Specifically, emotion analysis is performed using the IBM Watson Emotion Analysis API to generate optimal recommendations.

[1995] Step 12: Make a reservation at the salon and submit the information

[1996] The user makes a reservation at a hair salon based on the final simulation results. The input is the reservation information and the final simulation results, and the output is the reservation information and image data sent to the server. Specifically, the user selects the desired date and time and hair salon on the reservation page within the app, and that information is sent to the server.

[1997] Step 13: Transfer to salon device

[1998] The server transfers the received reservation information and image data to the salon terminal. The input is the reservation information and image data, and the output is the information sent to the salon terminal. Specifically, the data is transferred to the salon terminal using a backend service based on Java Spring Boot.

[1999] Step 14: Projecting onto the AI ​​mannequin

[2000] The salon terminal receives the simulation results and projects them onto an AI mannequin for the stylist to check. The input is the simulation result data, and the output is the simulated image projected onto the AI ​​mannequin. Specifically, a dedicated display and AI mannequin are used to project the simulation results.

[2001] Step 15: Treatment proposal based on emotion analysis results

[2002] Based on the analysis results of the emotion engine, the salon terminal will propose treatments that will satisfy the user. The input is the emotion analysis results, and the output is treatment proposal information. Specifically, the stylist will provide the user with the optimal treatment plan while referring to the emotion analysis results.

[2003] Step 16: Treatment

[2004] The stylist at the hair salon performs the treatment based on the received image and emotional data. The input is the final simulation image and emotional data, and the output is the hairstyle that will be applied to the user. Specifically, the AI ​​performs the actions of cutting and coloring the user's hair based on the simulation on the mannequin.

[2005] (Application example 2)

[2006] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[2007] In conventional hairstyle and hair color simulation systems, it was difficult for users to not only simulate hairstyles but also receive appropriate suggestions based on their emotions. Furthermore, there were no examples of virtual stores where users could easily share simulation results and then immediately purchase or make reservations. This meant that users could not fully confirm the satisfaction and suitability of the product before actually purchasing, potentially resulting in a dissatisfying experience.

[2008] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[2009] In this invention, the server includes an image input means that allows a user to simulate hairstyles and hair colors for an image of the user, an image processing means that generates an image of the simulated hairstyle and hair color, a display means that allows the user to check the generated image, an information transmission means that shares the generated image with the hair salon when making a reservation at the hair salon, a display means that allows the hair salon to display and check the received image, an emotion analysis means that analyzes the user's emotions and makes optimal suggestions based on the results, an image regeneration means that regenerates an optimal simulation based on the user's emotion data, and a purchase linkage means that allows the user to purchase items that the user likes based on the try-on simulation results. This allows the user to receive appropriate suggestions that match their emotions based on the simulation results, and further allows them to try on simulated items in a virtual store and directly purchase their favorite items.

[2010] "Image input means" means a device or interface that allows a user to take or upload an image of themselves.

[2011] "Image processing means" refers to algorithms or software for applying and simulating hairstyles and hair colors to photographed or uploaded images.

[2012] "Display means" refers to a device or screen that shows the generated simulation image to the user.

[2013] The "information transmission means" is a communication means for transmitting the generated simulation image to a beauty salon or other service provider.

[2014] An "emotion analysis method" is an algorithm or software that analyzes a user's emotions and makes optimal suggestions based on the results.

[2015] The "image regeneration means" is an algorithm or software for regenerating the simulation based on the results of the emotion analysis.

[2016] The "purchase linking method" is a function that allows users to purchase items they like on the spot based on the results of a try-on simulation.

[2017] "Image processing means" refers to algorithms or software used to process a user's image for anonymity.

[2018] The "video generation means" is an algorithm or software for generating a simulation image in video format so that the user can check the operation simulation.

[2019] A "virtual try-on means" is an interface or software that allows a user to simulate trying on clothes using a photograph of the user's face or whole body.

[2020] To implement this invention, it is necessary to build a system in which the user terminal, server, virtual store system, and beauty salon terminal all function in cooperation with each other. This system is composed of the following elements:

[2021] User Device

[2022] User terminals include smartphones, smart glasses, head-mounted displays, etc. They have the following functions:

[2023] Image input method: An interface that allows a user to take or upload a face or full-body photo, for example by selecting an image from a smartphone camera or photo gallery.

[2024] Display means: A screen for displaying simulation results and proposals to users. Simulation images and videos can be viewed.

[2025] Emotion input means: An interface for analyzing user emotions. Uses an AI engine (e.g., IBM Watson, Microsoft Azure Emotion API).

[2026] server

[2027] The server has the necessary hardware and software to perform the following functions:

[2028] Image processing means: Algorithms (e.g., OpenCV, TensorFlow) for applying hairstyles and hair colors to user-submitted images.

[2029] Image processing methods: Algorithms used to anonymize user images (e.g., image blurring or masking).

[2030] Video generation means: Software for converting simulation results into video format and providing operational simulation.

[2031] Sentiment analysis means: Equipped with a function to analyze emotional data and optimize suggestions (e.g., IBM Watson, Microsoft Azure).

[2032] Image regeneration means: An algorithm that regenerates the simulation based on the results of emotion analysis.

[2033] Purchase linkage: A feature that allows users to purchase items they like on the spot.

[2034] The server analyzes the image data received from the user and performs the following data processing and calculations:

[2035] Image analysis: Identify facial and body features using an AI engine (e.g., TensorFlow).

[2036] Apply Hairstyle: Apply the selected hairstyle and hair color to the image.

[2037] Sentiment Analysis: Analyzes emotional data using a sentiment analysis engine to generate optimal suggestions.

[2038] Simulation Regeneration: Regenerate the simulation based on the analysis results.

[2039] Display of simulation results: Generated images and videos are sent back to the user's device.

[2040] Virtual store system

[2041] The virtual store system implements the following functions:

[2042] Virtual try-on tool: An interface for simulating trying on clothes using a photo of the user's face or full body.

[2043] Purchase linking method: Provide users with a link to purchase items they like based on the simulation results.

[2044] Beauty salon terminal

[2045] The salon terminal includes computers and tablets used by salons. It has the following functions:

[2046] Display means: Simulation images and videos received from the server are displayed and used as a reference for treatment.

[2047] Information transmission method: A function for checking reservation information and simulation results and reflecting them in treatment.

[2048] Specific examples

[2049] The processing flow will be explained below with specific examples.

[2050] 1. Image acquisition: Take a photo of your face with your smartphone and upload it to the app.

[2051] 2. Image analysis: Images sent to the server are analyzed using an AI engine (TensorFlow).

[2052] 3. Simulation Generation: Apply the selected hairstyle and color to the image and generate the result.

[2053] 4. Sentiment analysis: The user inputs their feelings about the generated image, which is then analyzed by the sentiment analysis engine.

[2054] 5. Regeneration: Based on the results of sentiment analysis, the optimal simulation is regenerated.

[2055] 6. Result display: Display the simulation results to the user and readjust if necessary.

[2056] 7. Purchasing integration: Try on items in a virtual store system and purchase your favorite items on the spot.

[2057] Prompt Sentence Examples

[2058] Example prompts for generative AI models:

[2059] "Please simulate applying items selected from the clothing list below to a face photo taken or uploaded by the user. Please combine the items from the provided list with the user's image and return the results. Furthermore, please analyze the user's emotional data and suggest optimal items based on their emotions. Please generate a final image containing the suggested results."

[2060] In this way, the present invention provides a comprehensive system that allows users to simulate hairstyles and fashions that are more suitable for them, and by adding emotion analysis and virtual try-on functions, it is possible to increase user satisfaction.

[2061] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[2062] Step 1:

[2063] Users launch the app using a smartphone, smart glasses, or head-mounted display and take or upload a photo of their face or full body.

[2064] Input: A face or full-body photo of the user.

[2065] Output: Image data captured or uploaded.

[2066] Specific behavior: Launch the camera app and take a photo, or select an image from your photo gallery.

[2067] Step 2:

[2068] The user terminal transmits the captured or uploaded image data to the server.

[2069] Input: Image data captured or uploaded by the user.

[2070] Output: Image data sent to the server.

[2071] What it does: Encodes image data and uploads it to a server via a secure communication protocol.

[2072] Step 3:

[2073] The server analyzes the received image data using an AI engine (e.g., TensorFlow) to identify facial and physical features.

[2074] Input: Image data sent to the server.

[2075] Output: Analyzed facial and body feature data.

[2076] How it works: Image data is fed into the AI ​​model, feature points are extracted, and facial recognition algorithms are applied to identify facial contours and features.

[2077] Step 4:

[2078] The server applies the user's selected hairstyle and color to the image to generate a simulation.

[2079] Input: Analyzed facial and body feature data and user-selected hairstyle and color information.

[2080] Output: Image data of the simulation results.

[2081] What it does: Based on facial feature points, it applies the selected hairstyle and color to generate a realistic simulation image.

[2082] Step 5:

[2083] The server transmits the generated simulation image to the user terminal and displays it.

[2084] Input: Image data of the simulation results.

[2085] Output: The simulated image displayed on the user's device.

[2086] Specific operation: Encode the simulated image and send it to the user's device, where it is decoded and displayed on the screen.

[2087] Step 6:

[2088] The user checks the simulation results and inputs his / her feelings about the results using the emotion input means.

[2089] Input: Simulation result display and user emotion data.

[2090] Output: The input information, including emotion data.

[2091] Specific behavior: Look at the simulation results, make decisions, and operate the interface for emotion input.

[2092] Step 7:

[2093] The server analyzes the emotional data using an emotion analysis engine (e.g., IBM Watson) and generates optimal suggestions.

[2094] Input: User emotion data.

[2095] Output: Data with the best suggestions.

[2096] What it does: Passes input data to the sentiment analysis engine, retrieves the analysis results, and generates appropriate suggestions.

[2097] Step 8:

[2098] The server regenerates the simulation based on the emotion analysis results and transmits it again to the user terminal.

[2099] Input: Sentiment analysis results and original simulated image data.

[2100] Output: Regenerated simulated image data.

[2101] Specific operation: A new simulation is generated using hairstyles and colors adjusted based on the results of emotion analysis and transmitted to the user's device.

[2102] Step 9:

[2103] Users can simulate trying on items using a virtual fitting system and purchase the items they like on the spot.

[2104] Input: Regenerated simulation image and user selection information.

[2105] Output: Information about the completed purchase.

[2106] Specific actions: Operate the try-on simulation in the virtual try-on system and use the interface to complete the purchase process.

[2107] The above is a specific processing flow for carrying out the present invention.

[2108] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the control target 443 to output the result of the specific processing. The microphone 238 acquires voice indicating a user input regarding the result of the specific processing. The control unit 46A transmits voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[2109] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[2110] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the robot 414.

[2111] The emotion identification model 59 as an emotion engine may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to an emotion map (see FIG. 9), which is a specific mapping. Similarly, the emotion identification model 59 may determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[2112] FIG. 9 is a diagram illustrating an emotion map 400 on which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. Emotions closer to the center of the concentric circles are more primitive. Emotions representing states and actions arising from a state of mind are arranged on the outer edges of the concentric circles. The concept of emotion includes both affect and mental states. Emotions generally generated from reactions occurring in the brain are arranged on the left side of the concentric circles. Emotions generally induced by situational judgment are arranged on the right side of the concentric circles. Emotions generally generated from reactions occurring in the brain and induced by situational judgment are arranged on the upper and lower sides of the concentric circles. Furthermore, the emotion of "pleasure" is arranged on the upper side of the concentric circles, and the emotion of "discomfort" is arranged on the lower side. In this way, in the emotion map 400, multiple emotions are mapped based on the structure by which emotions are generated, and emotions that tend to occur simultaneously are mapped close to each other.

[2113] These emotions are distributed in the 3 o'clock direction on emotion map 400, and typically fluctuate between relief and anxiety. In the right half of emotion map 400, situational awareness dominates over internal sensations, resulting in a sense of calm.

[2114] The inside of emotion map 400 represents what is going on in the mind, and the outside of emotion map 400 represents behavior, so the further you go outside emotion map 400, the more visible the emotions become (the more they are expressed in behavior).

[2115] Human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. Emotions can also be created for robots, automobiles, and motorcycles, based on various balances, such as posture and remaining battery life. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. An emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on Voice Emotion Recognition and Emotional Brain Physiological Signal Analysis Systems, Tokushima University, Doctoral Dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map lists emotions belonging to the "reaction" domain, where sensation is dominant. The right half of the emotion map lists emotions belonging to the "situation" domain, where situational awareness is dominant.

[2116] The emotion map defines two emotions that promote learning. One is a negative emotion on the situation side, around the middle of "repentance" or "reflection." In other words, this occurs when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is a positive emotion on the response side, around "desire." In other words, this occurs when the robot experiences positive feelings such as "I want more" or "I want to know more."

[2117] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​indicating each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple pieces of training data that are combinations of user input and emotion values ​​indicating each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions that are located close to each other have similar values, as in the emotion map 900 shown in FIG. 10. FIG. 10 shows an example in which multiple emotions, "relieved," "calm," and "reassuring," have similar emotion values.

[2118] The system according to the present disclosure has been described above mainly with respect to the functions of the data processing device 12, but the system according to the present disclosure is not necessarily implemented on a server. The system according to the present disclosure may be implemented as a general information processing system. The present disclosure may be implemented, for example, as a software program running on a personal computer or an application running on a smartphone, etc. The method according to the present disclosure may be provided to users in the form of SaaS (Software as a Service).

[2119] In the above embodiment, an example was given in which the specific processing is performed by one computer 22, but the technology of the present disclosure is not limited to this, and the specific processing may be distributed and performed by a plurality of computers including the computer 22. For example, the data generation model 58 may be provided in an external device of the data processing device 12, and data may be generated in the external device in accordance with input data.

[2120] In the above embodiment, an example in which the specific processing program 56 is stored in the storage 32 has been described, but the technology of the present disclosure is not limited to this. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-transitory storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-transitory storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes the specific processing in accordance with the specific processing program 56.

[2121] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[2122] It is not necessary to store all of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store all of the specific processing program 56 in the storage 32; only a portion of the specific processing program 56 may be stored.

[2123] The hardware resource for executing a specific process can be any of the following processors: An example of a processor is a CPU, which is a general-purpose processor that functions as a hardware resource for executing a specific process by executing software, i.e., a program. Another example of a processor is a dedicated electrical circuit, such as an FPGA (Field-Programmable Gate Array), a PLD (Programmable Logic Device), or an ASIC (Application Specific Integrated Circuit), which is a processor with a circuit configuration designed specifically for executing a specific process. Each processor has built-in or connected memory, and each processor uses the memory to execute the specific process.

[2124] The hardware resource that executes the specific processing may be configured with one of these various processors, or may be configured with a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Also, the hardware resource that executes the specific processing may be a single processor.

[2125] As an example of a system configured with a single processor, first, one processor is configured by combining one or more CPUs and software, and this processor functions as a hardware resource that executes a specific process. Second, there is a system that uses a processor that realizes the functions of an entire system including multiple hardware resources that execute a specific process on a single IC chip, as typified by SoC (System-on-a-chip). In this way, a specific process is realized using one or more of the above-mentioned various processors as hardware resources.

[2126] Furthermore, the hardware structure of these various processors can be, more specifically, an electric circuit that combines circuit elements such as semiconductor devices. The specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps may be deleted, new steps may be added, or the processing order may be rearranged, without departing from the spirit of the invention.

[2127] The above-described description and illustrations are a detailed explanation of the parts related to the technology of the present disclosure and are merely an example of the technology of the present disclosure. For example, the above description of the configuration, functions, actions, and effects is an explanation of an example of the configuration, functions, actions, and effects of the parts related to the technology of the present disclosure. Therefore, it goes without saying that unnecessary parts may be deleted, new elements may be added, or replacements may be made to the above-described description and illustrations within the scope of the gist of the technology of the present disclosure. Furthermore, to avoid confusion and facilitate understanding of the parts related to the technology of the present disclosure, the above-described description and illustrations omit explanations of common technical knowledge that do not require particular explanation to enable the implementation of the technology of the present disclosure.

[2128] All publications, patent applications, and technical standards mentioned in this specification are herein incorporated by reference to the same extent as if each individual publication, patent application, or technical standard was specifically and individually indicated to be incorporated by reference.

[2129] The following is further disclosed regarding the above embodiment.

[2130] (Claim 1)

[2131] an image input means for allowing a user to simulate hairstyles and hair colors for an image of the user;

[2132] image processing means for generating an image of a simulated hairstyle and hair color;

[2133] a display means for allowing the user to view the generated image;

[2134] an information transmission means for sharing the image generated when making a reservation at a beauty salon with the beauty salon;

[2135] a display means for the beauty salon to display and confirm the received image;

[2136] A system including:

[2137] (Claim 2)

[2138] An image processing means for anonymizing a user's facial photograph;

[2139] an information transmission means for transmitting the anonymized image to the beauty salon;

[2140] The system of claim 1 further comprising:

[2141] (Claim 3)

[2142] A video generation means for allowing the user to check the simulated hairstyle and hair color image in a video;

[2143] 10. The system of claim 1, further comprising the ability to simulate color changes due to seasons and weather.

[2144] "Example 1"

[2145] (Claim 1)

[2146] an image input means for allowing a user to simulate hairstyles and hair colors for an image of the user;

[2147] image processing means for generating an image of a simulated hairstyle and hair color;

[2148] a display means for allowing the user to view the generated image;

[2149] an information transmission means for sharing the image generated when making a reservation at a beauty salon with the beauty salon;

[2150] a display means for the beauty salon to display and confirm the received image;

[2151] a means for projecting the received image onto the AI ​​mannequin;

[2152] a face detection means for extracting a face portion of a user;

[2153] A video generation means that allows you to check the simulation results in video format;

[2154] A system including:

[2155] (Claim 2)

[2156] An image processing means for anonymizing a user's facial photograph;

[2157] an information transmission means for transmitting the anonymized image to the beauty salon;

[2158] 10. The system of claim 1.

[2159] (Claim 3)

[2160] A function to simulate color changes due to seasons and weather,

[2161] data compression means for compressing and transmitting the generated image;

[2162] 10. The system of claim 1.

[2163] "Application Example 1"

[2164] (Claim 1)

[2165] an image input means for allowing a user to simulate hairstyles and hair colors for an image of the user;

[2166] image processing means for generating an image of a simulated hairstyle and hair color;

[2167] a display means for allowing the user to view the generated image;

[2168] an information transmission means for sharing the image generated when making a reservation at a beauty salon with the beauty salon;

[2169] a display means for the beauty salon to display and confirm the received image;

[2170] A server that receives facial photos taken or uploaded by the user's device and analyzes them using an AI module;

[2171] a server that applies the selected hairstyle and hair color to a photograph of the user's face and returns the generated image;

[2172] A system including:

[2173] (Claim 2)

[2174] An image processing means for anonymizing a user's facial photograph;

[2175] an information transmission means for transmitting the anonymized image to the beauty salon;

[2176] An AI module that receives a user's face photo and generates a simulation result of hairstyle and color in real time.

[2177] The system of claim 1 further comprising:

[2178] (Claim 3)

[2179] A video generation means for allowing the user to check the simulated hairstyle and hair color image in a video;

[2180] 10. The system of claim 1, further comprising the ability to simulate color changes due to seasons and weather.

[2181] An interface that allows users to view simulation results in real time and make fine adjustments if necessary;

[2182] 10. The system of claim 1, comprising:

[2183] "Example 2: Combining Emotion Engines"

[2184] (Claim 1)

[2185] an image input means for allowing a user to simulate hairstyles and hair colors for an image of the user;

[2186] image processing means for generating an image of a simulated hairstyle and hair color;

[2187] a display means for allowing the user to view the generated image;

[2188] an information transmission means for sharing the image generated when making a reservation at a beauty salon with the beauty salon;

[2189] a display means for the beauty salon to display and confirm the received image;

[2190] An emotion input means for recognizing the emotion of a user;

[2191] An emotion engine that analyzes user emotions and makes optimal suggestions based on emotional data,

[2192] A system including:

[2193] (Claim 2)

[2194] An image processing means for anonymizing a user's facial photograph;

[2195] an information transmission means for transmitting the anonymized image to the beauty salon;

[2196] The system of claim 1 further comprising:

[2197] (Claim 3)

[2198] A video generation means for allowing the user to check the simulated hairstyle and hair color image in a video;

[2199] 10. The system of claim 1, further comprising the ability to simulate color changes due to seasons and weather.

[2200] "Application example 2 when combining emotion engines"

[2201] (Claim 1)

[2202] an image input means for allowing a user to simulate hairstyles and hair colors for an image of the user;

[2203] image processing means for generating an image of a simulated hairstyle and hair col...

Claims

1. an image input means for allowing a user to simulate hairstyles and hair colors for an image of the user; image processing means for generating an image of a simulated hairstyle and hair color; a display means for allowing the user to view the generated image; an information transmission means for sharing the image generated when making a reservation at a beauty salon with the beauty salon; a display means for the beauty salon to display and confirm the received image; A system including:

2. An image processing means for anonymizing a user's facial photograph; an information transmission means for transmitting the anonymized image to the beauty salon; The system of claim 1 further comprising:

3. A video generation means for allowing the user to check the simulated hairstyle and hair color image in a video; 10. The system of claim 1, further comprising a function for simulating color changes due to seasons and weather.

Images