System

Abstract

A system is provided.SOLUTION: A system comprising: means for receiving an image photographed or selected by a user; means for inputting an instruction of a hairstyle or a color desired by the user; means for processing the image in accordance with an instruction content of the user based on the received image using generative artificial intelligence; means for displaying the processed image to the user, receiving a fine adjustment instruction from the user, and processing the image again; and means for storing a final processed image and transmitting the final processed image to a hairdresser.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] When customers visit a beauty salon, it is often difficult for them to accurately communicate their ideal hairstyle and color to the hairdresser, resulting in the finished product not meeting their expectations. A solution to this problem is needed to resolve the causes of this low customer satisfaction. [Means for solving the problem]

[0005] The present invention provides a system that uses generative artificial intelligence to accurately communicate a user's ideal hairstyle and color. Specifically, the system includes a means for receiving an image taken or selected by the user, a means for inputting instructions for the user's desired hairstyle and color, a means for processing the image based on the received image using generative artificial intelligence to match the user's instructions, a means for displaying the processed image to the user, receiving fine-tuning instructions from the user and processing it again, and a means for saving the final processed image and sending it to the hairdresser. This system prevents misunderstandings between the user and the hairdresser and improves user satisfaction.

[0006] "User" refers to an individual who uses the system to specify their ideal hairstyle and color.

[0007] "Image" refers to photographic data taken or selected by the user that can be used to edit hairstyle or color.

[0008] "Instructions" are information detailing the user's desired hairstyle and color, and refer to the input content that is interpreted by the generative AI.

[0009] "Generative AI" refers to algorithms and systems that process hairstyles and colors based on user images and instructions.

[0010] "Enhancing" refers to the process of modifying or transforming an image to a user-specified hairstyle or color.

[0011] "Interface" refers to the screen or input device through which the user inputs instructions, checks the processed image, and gives instructions for further fine-tuning.

[0012] "Storage" refers to the act of temporarily or permanently recording the final processed image in a database or storage.

[0013] "Sending" refers to the act of the system transferring the processed image to the hairdresser.

[0014] A "hairdresser" is a professional who actually applies hairstyles and colors based on the user's instructions.

[0015] "Receipt confirmation" refers to the process by which the server confirms that the image has been received by the hairdresser to whom it was sent.

[0016] "Feedback" refers to the act of providing users with information regarding transmission status and reception results.

[0017] "Database" refers to an information collection system for managing and storing information such as images, instructions, and transmission history within the system. [Brief explanation of the drawings]

[0018] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10]1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 3 is a sequence diagram showing a processing flow of the data processing system according to the first embodiment. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0019] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0020] First, the terms used in the following description will be explained.

[0021] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0022] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0023] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0024] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0025] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0026] [First embodiment]

[0027] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0028] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0029] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0030] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0031] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0032] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0033] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0034] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0035] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0036] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0037] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0038] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0039] The present invention is a system that allows users to accurately communicate their ideal hairstyle and color when visiting a beauty salon. This system operates in cooperation with the user, terminal, and server elements. A specific example is shown below.

[0040] User Actions

[0041] The first step in using this system is for the user to install and launch the dedicated application. Next, the user takes a photo of their face or selects a photo they have already taken from their gallery. The selected photo is uploaded within the app. This upload action causes the device to send the user's photo to the server.

[0042] Server Processing

[0043] The server receives the photos sent from the device and temporarily stores them. The user uses a chat interface to input instructions for the desired hairstyle and color. The user's instructions are sent to the server via the device, and the server analyzes them. The analyzed instructions are then provided to a generation AI, which then generates an image with the hairstyle and color modified based on the user's specifications.

[0044] Image generation and display

[0045] The image processed by the generation AI is sent from the server to the device and displayed for the user to review. The user reviews the processed image and inputs fine-tuning instructions as needed. These fine-tuning instructions are sent back to the server, which then requests further processing from the generation AI. Fine-tuning is carried out and a new image is generated. This process is repeated until the user is satisfied.

[0046] Final confirmation and submission

[0047] If the user is satisfied with the final image, they tap the "OK" button. This causes the device to prepare to send the final processed image to the server, which then forwards the image to the designated hairdresser. The hairdresser then confirms receipt, and this confirmation information is sent back to the user via the server.

[0048] Specific examples

[0049] As a concrete example, consider a scenario in which a user launches a dedicated app and selects and uploads a selfie. After this photo is sent to the server, the user uses a chat interface to input instructions such as, "Shoulder-length hair, swept-back bangs, and ash gray color." The server analyzes these instructions and sends a processing request to the generation AI. The generation AI generates an image based on the user's face photo with the hairstyle and color applied, and sends this back to the user. After the user confirms this, they can further input fine-tuning instructions such as, "Make the bangs a little shorter." The server processes the image again using the generation AI, and finally sends the final image they are satisfied with to the hairdresser. As a result, the user can accurately communicate their ideal hairstyle and achieve the desired results.

[0050] The processing flow will be explained below.

[0051] Program processing steps

[0052] 1. User photo submission

[0053] Step 1: Select and upload a photo

[0054] User

[0055] Open the application.

[0056] Tap the "Select Photo" button to choose a photo of your face from your device's gallery or take a new one.

[0057] Tap the "Upload" button to send the photo you selected or took.

[0058] Step 2: Prepare your photos for sending

[0059] Terminal

[0060] Temporarily store photos uploaded by users.

[0061] Sends a request to the server to prepare the photo for sending.

[0062] Step 3: Receive and save photos

[0063] server

[0064] Receives photo data sent from the device.

[0065] Save the photos in a database.

[0066] A confirmation to save the photo will be sent to your device.

[0067] ---

[0068] 2. Specify hairstyle and color

[0069] Step 4: Enter instructions

[0070] User

[0071] Use the app's chat interface to input your desired cut (e.g., "shoulder-length") and color (e.g., "ash gray").

[0072] Tap the "Send" button to send the instructions.

[0073] Step 5: Sending instructions

[0074] Terminal

[0075] Receive and temporarily store instructions from the user.

[0076] The instruction is sent to the server.

[0077] Step 6: Parse instructions and input them into the generative AI

[0078] server

[0079] Analyze the instructions received from the terminal.

[0080] The saved photos and instructions are input into the generative AI system.

[0081] The generative AI generates images with edited hairstyles and colors based on the user's instructions.

[0082] ---

[0083] 3. Instructions for fine-tuning and their reflection

[0084] Step 7: Check the processed image and give instructions for fine adjustments

[0085] User

[0086] The processed image returned from the server is checked within the application.

[0087] If you need to make any tweaks, enter new instructions (e.g., "Make the bangs a little shorter") and tap the "Send" button.

[0088] Step 8: Sending fine-tuning instructions

[0089] Terminal

[0090] Receives new instructions and sends them to the server.

[0091] Step 9: Applying the tweaks

[0092] server

[0093] Re-input the received fine-tuning instructions into the generation AI.

[0094] The generative AI readjusts the image and generates a new one.

[0095] The adjusted image is sent to the device.

[0096] This process is repeated until the user is satisfied.

[0097] ---

[0098] 4. Sending the final image

[0099] Step 10: Check the final image

[0100] User

[0101] Finally, check the image of the hairstyle you are satisfied with in the application.

[0102] Tap the "Decide Now" button.

[0103] Step 11: Prepare your images for delivery

[0104] Terminal

[0105] Prepare the final processed image for sending to the beauty salon.

[0106] Step 12: Send to salon

[0107] server

[0108] Send a message to your salon contact with the final processed image.

[0109] The hairdresser's confirmation of receipt is obtained and the status is fed back to the terminal.

[0110] Save the transmission history in the database.

[0111] Example 1

[0112] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0113] When visiting a hair salon, it is difficult for users to accurately communicate their ideal hairstyle and color. In particular, a communication gap between the user and the hairdresser makes it difficult for users to achieve a satisfactory result. For this reason, a system is needed that allows users to visually confirm their ideal style and accurately communicate it to the hairdresser.

[0114] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0115] In this invention, the server includes a means for receiving an image taken or selected by the user, a means for inputting instructions for the user's desired hairstyle and color, a means for processing the image based on the received image in accordance with the user's instructions using a generative artificial intelligence, a means for displaying the processed image to the user, receiving fine adjustment instructions from the user and processing it again, and a means for saving the final processed image and sending it to the hairdresser. This allows the user to visually confirm their ideal hairstyle and color and accurately communicate them to the hairdresser.

[0116] A "user" is an individual who uses the system to specify their ideal hairstyle and color and actually generate an image.

[0117] "Photo or Selected Image" means an image of a user's face taken or selected from existing images and uploaded to the system.

[0118] "Instructions" refers to the specific input the user makes about the hairstyle and color they desire.

[0119] "Generative AI" refers to an AI technology for processing images based on user instructions, such as generative models.

[0120] "Means for processing images" refers to the functionality for using the generated AI model to edit or modify images taken or selected based on user instructions.

[0121] "Fine-tuning instructions" refer to requests for additional corrections or changes made by the user to the displayed edited image.

[0122] The "final edited image" refers to the final image that the user is satisfied with after making fine adjustments.

[0123] A "hairdresser" is a professional who receives the final edited image sent by the user and performs the actual hairstyle and coloring based on it.

[0124] "Server" refers to the core computer system that receives and stores user images, analyzes instructions, and works with the generating AI.

[0125] The present invention is a system that allows users to accurately communicate their ideal hairstyle and color when visiting a beauty salon. This system achieves its purpose by having the user, terminal, and server work together.

[0126] User Actions

[0127] Users install and launch the dedicated application from the App Store or GOOGLE PLAY®. After creating an account and logging in, they can take a selfie using the app's photo feature or select an existing photo from their gallery. After selection, the device uploads the photo to the server. For example, if a user takes a selfie and taps the upload button in the app, the device compresses the photo and sends it to the server using a secure protocol.

[0128] Server Processing

[0129] The server receives the image sent by the user and temporarily stores it. The user then inputs their desired hairstyle and color through the chat interface. For example, they might input specific instructions such as "shoulder-length hair, swept-back bangs, and ash gray color." The instructions are sent via the user's device to the server, which analyzes them. The analyzed information is then provided to the generation AI, which then processes the image based on the user's instructions.

[0130] Examples of generative AI that can be used include Stable Diffusion and DALL-E. The server sends appropriate prompts to the generative AI model to generate images. For example, the following prompts are used:

[0131] "Generate an image of the user's face with shoulder-length hair, swept-back bangs, and ash gray hair."

[0132] Image generation and display

[0133] The image processed by the generation AI is sent from the server to the device and displayed for the user to review. At this point, the user can review the generated image and input fine-tuning instructions as needed. For example, they can input fine-tuning instructions such as "Make the bangs a little shorter." The instructions sent back to the server are analyzed, and a new image is generated by sending additional prompts to the generation AI. The image is then sent to the user again, and this process is repeated until the user is satisfied.

[0134] Final confirmation and submission

[0135] When the user finally hits the "OK" button on the image they are satisfied with, the device prepares to send the final processed image to the server, which then forwards the image to a dedicated hairdresser. After the hairdresser confirms receipt, the confirmation information is sent back to the user via the server.

[0136] This system allows users to visually confirm their ideal hairstyle and color and accurately communicate it to the hairdresser, which is expected to result in the user achieving the desired results.

[0137] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0138] Step 1: Initial Setup

[0139] The user installs and launches the dedicated application, creates an account, and logs in.

[0140] Input: Installed apps, user information (email address, password)

[0141] Output: The app launches and the user is logged in.

[0142] Step 2: Upload your photo

[0143] The user takes a photo of their face using the app's photo function or selects an existing photo, and the device uploads the selected photo to the server.

[0144] Input: Captured or selected image

[0145] Output: The image is uploaded to the server.

[0146] Specific operation: For example, when you tap the camera button in the app, the camera starts up, you take a picture, check it, and then tap the upload button. The device compresses the image and sends it to the server using the HTTPS protocol.

[0147] Step 3: User prompts

[0148] Users input their desired hairstyle and color through a chat interface, and the device sends the user's instructions to the server.

[0149] Input: User's hairstyle and color instructions (e.g., "Shoulder-length, swept-back bangs, ash gray color")

[0150] Output: The instructions are sent to the server.

[0151] Specific operation: The user enters text in the chat interface and taps the send button. The device sends the text data to the server.

[0152] Step 4: Instruction analysis and image generation

[0153] The server receives and analyzes the user's instructions. The server sends the analysis results to the generation AI, which issues instructions as prompts. The generation AI generates an image based on the prompts.

[0154] Input: User's instructions (text), user's image data

[0155] Output: processed image

[0156] Specific operation: The server uses a natural language processing engine to analyze the instructions, generate specific prompts such as "Hair length: shoulder length," "Bangs: swept-back," and "Color: ash gray," and sends them to the generation AI. The generation AI then generates an image based on this and sends it back to the server.

[0157] Step 5: Image generation and display

[0158] The image generated by the AI ​​is received and stored by the server, which then sends the image to the device and displays it for the user to view.

[0159] Input: Generated image

[0160] Output: The image is displayed on the user's device.

[0161] Specific operation: The server temporarily saves the generated image file in a database, generates a URL and sends it to the device, where the image is displayed and the user is asked to confirm it.

[0162] Step 6: Fine-tuning instructions

[0163] The user checks the generated image and inputs fine-tuning instructions as needed. The device then sends the fine-tuning instructions to the server.

[0164] Input: User fine-tuning instructions (e.g., "Make my bangs a little shorter")

[0165] Output: Fine-tuning instructions are sent to the server.

[0166] Specific operation: The user enters new instructions in the chat interface and taps the send button. The device sends the text data to the server again.

[0167] Step 7: Reprocessing and display

[0168] The server then sends a new prompt to the AI ​​generator, requesting it to generate a finely tuned image. The AI ​​generator generates a new image, which the server stores and sends to the device.

[0169] Input: New fine-tuning instructions, original image data

[0170] Output: New processed image

[0171] What it does: The server analyzes the new instructions and sends prompts to the generation AI to generate new image data, which is then sent back to the device and displayed to the user.

[0172] Step 8: Final Review and Submission

[0173] If the user is satisfied with the final image, they tap the "OK" button. The device prepares and sends the final image to the server. The server then forwards the final image to the designated hairdresser. The hairdresser confirms receipt, and the server returns the confirmation information to the user.

[0174] Input: Final processed image

[0175] Output: Final image is sent to the hairdresser, confirmation information is fed back to the user.

[0176] Specific operation: When the user taps the "OK" button, the device compresses the final image data and sends it to the server. The server then forwards the image to the hairdresser, receives confirmation from the hairdresser, and sends feedback to the device.

[0177] (Application example 1)

[0178] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0179] There is a problem that it is difficult for hair salon customers to accurately communicate their ideal hairstyle and color to the hairdresser. This problem manifests itself in the form of miscommunication and dissatisfaction with the finished product due to differences in image. In addition, insufficient reservation management and customer service upon arrival are factors that detract from the customer experience at hair salons. To solve these issues, a system is needed that allows users to virtually try on their ideal hairstyle from home or on the go and share their specific image with the hairdresser.

[0180] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0181] In this invention, the server includes a means for receiving images taken or selected by the user, a means for inputting instructions for the user's desired hairstyle and color, a means for processing the received image according to the user's instructions using a generative AI model, a means for displaying the processed image to the user and receiving fine-tuning instructions from the user for further processing, a means for saving the final processed image and sending it to the hairdresser, and a means for managing the user's appointments. This allows users to try out their ideal hairstyle from home or on the go and share their specific image with the hairdresser, facilitating communication and improving satisfaction with the finished product. It also streamlines appointment management and customer service during visits, improving the overall customer experience.

[0182] "User" refers to an individual who uses the system to try out their ideal hairstyle and color and input their instructions.

[0183] "Image receiving means" refers to a device or system that has the function of uploading and receiving photos taken or selected by a user to a server through an application.

[0184] "Instruction input means" refers to a device or system having an interface for a user to input details of the hairstyle or color desired by the user via text or voice.

[0185] "Generative AI model" refers to an algorithm or system that uses artificial intelligence technology to generate or modify images based on information provided by users.

[0186] "Image processing means" refers to a device or system that edits and processes images in accordance with user instructions based on the received image using a generative AI model.

[0187] The term "fine-tuning instruction receiving means" refers to a device or system that has the function of displaying the processed image, receiving additional instructions from the user, and reprocessing the image based on those instructions.

[0188] "Final image storage means" refers to a device or system that stores the final processed image that the user is satisfied with.

[0189] The term "means for transmitting to a hairdresser" refers to a device or system that has the function of transmitting the final processed image to a designated hairdresser.

[0190] "Reservation management means" refers to a device or system that has the function of managing user reservation information and adjusting reservations and schedules.

[0191] "Reprocessing interface" refers to an interface that allows a user to review the generated image and input instructions for reprocessing, if necessary.

[0192] A "prompt" is a text that contains specific instructions for a generative AI model, and refers to an instruction used to improve the accuracy of image generation and processing.

[0193] "Professionals" refers to hairdressers and hairstylists who receive the images and instructions sent by the system and actually perform the hairstyle and coloring.

[0194] This invention is a system that can accurately convey the ideal hairstyle and color when visiting a beauty salon. This system works in cooperation with the user's device, a server, and a generative AI model.

[0195] User Actions

[0196] Users first install the app on their smartphone and launch it. Then, they take a photo of themselves or select a photo they have already taken from their gallery. The selected photo is then uploaded within the app and sent to the server.

[0197] Server Processing

[0198] The server receives the user's photo sent from the device and temporarily stores it. When the user inputs instructions for the desired hairstyle or color using the chat interface, the instructions are sent to the server via the device. The server analyzes the instructions and provides them as prompts to the generative AI model.

[0199] Image generation using generative AI models

[0200] The generative AI model processes the received image based on the user's instructions. For example, if the instruction is "short bob, black hair, swept bangs," the generative AI model will generate an image based on this information.

[0201] Example prompt sentence:

[0202] "Generate an image of a user with a short bob, dark hair, and swept-back bangs."

[0203] Image confirmation and fine adjustment

[0204] The processed image is sent from the server to the device and displayed for the user to review. The user can then confirm that the image is as desired and input fine-tuning instructions as needed. These fine-tuning instructions are sent back to the server, which requests further processing from the generative AI model. This process is repeated until the user is satisfied.

[0205] Final confirmation and submission

[0206] If the user is satisfied with the final image, they tap the "OK" button. The device prepares to send the final processed image to the server, which then forwards the image to the designated hairdresser. The hairdresser confirms receipt, and the confirmation information is sent back to the user via the server.

[0207] Reservation management and store visit support

[0208] In addition, the system is equipped with a function to manage user reservation information, which allows for smooth adjustment of user reservations and streamlines customer service when they visit the store.

[0209] Specific examples

[0210] For example, a user launches a dedicated app and selects and uploads a selfie. After the photo is sent to the server, the user uses a chat interface to input instructions such as, "I want shoulder-length hair, swept-back bangs, and an ash gray color." The server analyzes these instructions and sends a request to the generative AI model to edit the image. The generative AI model generates an image based on the user's face photo with the hairstyle and color applied, and sends this back to the user. After the user confirms the image, the user can again input instructions for fine-tuning, such as, "Make the bangs a little shorter." The server then requests further editing from the generative AI model and generates a new image. This process is repeated until the user is satisfied. The final image is sent to the hairdresser, and appointment information is managed, allowing the user to get the hairstyle they want.

[0211] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0212] Step 1:

[0213] The user installs and launches the dedicated app on their smartphone. Next, they take a photo of their face or select a previously taken photo from the gallery. The selected photo is uploaded within the app. The user's photo is provided as input, and the photo data is sent to the application server. The server then receives the user's image data.

[0214] Step 2:

[0215] The server receives the user's photos sent from the device and temporarily stores them. Once the photo data is received, it prepares to analyze it.

[0216] Step 3:

[0217] The user uses a chat interface to input instructions for the desired hairstyle and color. The instructions are provided in text format as input, and the instructions are sent from the device to a server, which parses the instructions and generates a prompt for the generative AI model.

[0218] Step 4:

[0219] The server sends the generated prompt to the generative AI model. Specifically, the prompt is used to make the generative AI model generate an image with the hairstyle and color applied. The prompt and photo data are provided as input, and the generative AI model outputs an edited image based on these.

[0220] Step 5:

[0221] The processed image generated by the generative AI model is sent from the server to the device, where the server receives the generated image and returns it in a format that can be displayed to the user.

[0222] Step 6:

[0223] The user checks the generated image, and if it is not as desired, inputs instructions for fine-tuning in the chat interface. The fine-tuning instructions are provided as input from the user, and the instruction data is sent from the terminal to the server.

[0224] Step 7:

[0225] The server receives the user's fine-tuning instructions and sends them to the generative AI model as a new prompt. The fine-tuning instructions are provided as input, and the generative AI model outputs a newly processed image.

[0226] Step 8:

[0227] The regenerated processed image is sent from the server to the user's device, where the user can review it again. This process is repeated until the user is satisfied.

[0228] Step 9:

[0229] If the user is satisfied with the final image, he / she presses the "OK" button, providing the final processed image as input, which is then sent to the server.

[0230] Step 10:

[0231] The server stores the final processed image and sends it to the hairdresser. The final processed image is provided as input and forwarded to the hairdresser. The hairdresser confirms receipt, and the confirmation information is fed back to the user through the server.

[0232] Step 11:

[0233] The system manages the user's reservation information and provides hairdressers with information to ensure a smooth customer experience when they visit. The user's reservation information is provided as input and shared with the hairdresser.

[0234] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0235] The present invention is an advanced system that combines a system that accurately conveys the ideal hairstyle and color when visiting a beauty salon with an emotion engine that recognizes the user's emotions. Specific embodiments executed by the user, terminal, and server are described below.

[0236] User Actions

[0237] Users install and launch the app, then take a photo of themselves or select a photo they have already taken from their gallery, and upload the photo within the app. This upload process causes the device to send the photo to the server.

[0238] Server Processing

[0239] The server receives the photos sent from the device and temporarily stores them. Users use a chat interface to input instructions for their desired hairstyle and color. The user's instructions are sent via the device to the server, which analyzes them. The analyzed instructions are then provided to a generation AI, which then generates an image with the hairstyle and color modified based on the user's specifications.

[0240] Image generation and display

[0241] The image processed by the generation AI is sent from the server to the device and displayed for the user to review. The user reviews the processed image and enters instructions for fine-tuning as needed. These instructions are sent back to the server, which then requests further processing from the generation AI. Fine-tuning is carried out and a new image is generated. This process is repeated until the user is satisfied.

[0242] The role of the emotional engine

[0243] As the user reviews the image, the emotion engine analyzes the user's facial expressions and voice to recognize their emotions. For example, if the user looks dissatisfied, the emotion engine automatically suggests an alternative. The emotion analysis results are sent to the server and reflected in the next step of processing.

[0244] Final confirmation and submission

[0245] When the user finally taps the "OK" button on the image they are satisfied with, the device prepares to send the final edited image to the salon. The server forwards this edited image to the salon's designated contact and obtains the hairdresser's confirmation of receipt. The results of the user's sentiment analysis are also sent to the hairdresser, allowing the hairdresser to understand the user's mood and preferences before the treatment. This information is then fed back to the user.

[0246] Specific examples

[0247] As a concrete example, we will show a scene in which a user launches a dedicated application and selects and uploads a selfie. After the photo is sent to the server, the user uses a chat interface to input instructions such as, "I want shoulder-length hair, swept-back bangs, and an ash gray color." The server analyzes these instructions and sends a processing request to the generation AI. The generation AI generates an image based on the user's face photo, applying the hairstyle and color, and sends this back to the user. When the user reviews the image, the emotion engine analyzes the user's facial expression. If the user is not satisfied, it automatically suggests, "Make the bangs a little shorter." The user accepts, and the server uses the generation AI to process the image again. This process is repeated, and finally, the image that the user is satisfied with is sent to the hairdresser. The hairdresser confirms receipt and performs the treatment taking the emotion analysis results into account. As a result, the user can accurately communicate their ideal hairstyle and achieve the desired results.

[0248] The processing flow will be explained below.

[0249] Program processing steps

[0250] 1. User photo submission

[0251] Step 1: Select and upload a photo

[0252] User

[0253] Open the application.

[0254] Tap the "Select Photo" button to choose a photo of your face from your device's gallery or take a new one.

[0255] Tap the "Upload" button to send the photo you selected or took.

[0256] Step 2: Prepare your photos for sending

[0257] Terminal

[0258] Temporarily store photos uploaded by users.

[0259] Sends a request to the server to prepare the photo for sending.

[0260] Step 3: Receive and save photos

[0261] server

[0262] Receives photo data sent from the device.

[0263] Save the photos in a database.

[0264] A confirmation to save the photo will be sent to your device.

[0265] ---

[0266] 2. Specify hairstyle and color

[0267] Step 4: Enter instructions

[0268] User

[0269] Use the app's chat interface to input your desired cut (e.g., "shoulder-length") and color (e.g., "ash gray").

[0270] Tap the "Send" button to send the instructions.

[0271] Step 5: Sending instructions

[0272] Terminal

[0273] Receive and temporarily store instructions from the user.

[0274] The instruction is sent to the server.

[0275] Step 6: Parse instructions and input them into the generative AI

[0276] server

[0277] Analyze the instructions received from the terminal.

[0278] The saved photos and instructions are input into the generative AI system.

[0279] The generative AI generates images with edited hairstyles and colors based on the user's instructions.

[0280] ---

[0281] 3. Instructions for fine-tuning and their reflection

[0282] Step 7: Checking edited images and analyzing sentiment

[0283] User

[0284] The processed image returned from the server is checked within the application.

[0285] During this process, the emotion engine analyzes the user's facial expressions and voice to recognize their emotions.

[0286] Step 8: Emotional Feedback

[0287] server

[0288] The emotion engine receives the user's recognized emotions and presents the next suggestion to the user based on the analysis results.

[0289] For example, if the person looks dissatisfied, the system will make a suggestion such as, "Would you like your bangs cut a little shorter?"

[0290] Step 9: Enter and submit fine-tuning instructions

[0291] User

[0292] Accept the suggestions based on the sentiment analysis results, or enter new fine-tuning instructions (e.g., "Make it a little brighter").

[0293] Send the instructions by tapping the "Send" button.

[0294] Step 10: Analyze and refine the fine-tuning instructions

[0295] server

[0296] Re-input the received fine-tuning instructions into the generation AI.

[0297] The generative AI readjusts the image and generates a new one.

[0298] The adjusted image is sent to the device.

[0299] This process is repeated until the user is satisfied.

[0300] ---

[0301] 4. Sending the final image

[0302] Step 11: Check the final image

[0303] User

[0304] Finally, check the image of the hairstyle you are satisfied with in the application.

[0305] Tap the "Decide Now" button.

[0306] Step 12: Prepare your images for delivery

[0307] Terminal

[0308] Prepare the final processed image for sending to the beauty salon.

[0309] Step 13: Send to salon

[0310] server

[0311] Send a message to your salon contact with the final processed image.

[0312] The hairdresser's confirmation of receipt is obtained and the status is fed back to the terminal.

[0313] Save the transmission history in the database.

[0314] Step 14: Sending sentiment analysis results to hairdressers

[0315] server

[0316] The results of the user's sentiment analysis are also sent to the hairdresser, allowing the hairdresser to understand the user's mood and preferences before the treatment.

[0317] Specific examples

[0318] As a concrete example, consider a scenario in which a user launches a dedicated application, selects and uploads a selfie. After the photo is sent to the server, the user enters instructions in the chat interface, such as "I want shoulder-length hair, swept-back bangs, and an ash gray color," and submits the image. The server analyzes the instructions and generates an edited image using generative AI. When the user reviews the image, the emotion engine suggests, "Maybe the bangs would be better if they were a little shorter." The user accepts the suggestion, and the server again fine-tunes the image using generative AI. The final image that the user is satisfied with is sent to the hairdresser, along with the results of the emotion analysis. This process allows the user to accurately communicate their ideal hairstyle and achieve the desired results.

[0319] Example 2

[0320] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0321] In conventional beauty salons, communication between customers and hairdressers has been hindered by the difficulty of accurately conveying the customer's desired hairstyle and color. Furthermore, if the customer is dissatisfied with the processed image, their feelings are not properly reflected, making it difficult to achieve the desired result. The present invention aims to solve these problems and provide a system that allows customers to accurately convey their desired hairstyle and color and achieve satisfactory results.

[0322] The specification process by the specification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes a means for receiving an image taken or selected by the user, a means for inputting instructions for the user's desired hairstyle and color, a means for processing the image based on the received image in accordance with the user's instructions using a generative artificial intelligence model, a means for displaying the processed image to the user and receiving fine-tuning instructions from the user for further processing, a means for saving the final processed image and sending it to the hairdresser, and a means for recognizing the user's emotions and automatically making suggestions based on the emotions. This allows the customer to accurately communicate their ideal hairstyle and color, ensure that their emotions are properly reflected in the generated image, and ultimately achieve a satisfactory result.

[0323] "User" refers to an individual who uses this system to give instructions about hairstyles and colors and check the results.

[0324] The "server" refers to the infrastructure responsible for processing user images and instructions, processing images using generative AI models, and recognizing user emotions.

[0325] A "generative AI model" is an artificial intelligence algorithm that processes hairstyles and colors based on a user's facial photo and instructions, specifically referring to technologies such as StyleGAN2.

[0326] An "emotion engine" refers to a software component that analyzes a user's facial expressions and voice and recognizes their emotions.

[0327] "Means for receiving images" refers to a function for transmitting an image taken or selected by a user to a server, and for the server to receive the image.

[0328] "Means for inputting instructions" refers to an interface through which a user can input information about the hairstyle and color they desire.

[0329] "Means of processing" refers to the function of processing received images based on user instructions using a generative AI model.

[0330] "Means for receiving fine-tuning instructions and reprocessing" refers to a function for receiving additional processing instructions from the user and reprocessing the image.

[0331] "Means for saving and sending to a hairdresser" refers to the function of saving the final processed image and sending the image to a hairdresser.

[0332] "Means for automatically making emotion-based suggestions" refers to the ability of the emotion engine to recognize the user's emotions and automatically make different suggestions based on the results.

[0333] "Final processed image" refers to the image generated in its final form, reflecting the user's wishes and fine-tuning instructions.

[0334] "Means for providing feedback" refers to features that notify users, such as acknowledgments from their hairdressers, to improve their experience.

[0335] The present invention is an advanced system that combines a system that accurately conveys the ideal hairstyle and color when visiting a beauty salon with an emotion engine that recognizes the user's emotions. Specific embodiments executed by the user, terminal, and server are described below.

[0336] User Actions

[0337] The user installs and launches a dedicated application on their smartphone. This application is available for iOS and ANDROID (registered trademark) platforms. The user takes a photo of their face using the application's camera function or selects a previously taken photo from the gallery. The selected photo is then uploaded within the app. This upload causes the device to send the photo to the server.

[0338] Terminal handling

[0339] The device converts the selected photo into a data packet and sends it over the Internet to a server. The server temporarily stores the image in an Amazon Web Services (AWS) S3 bucket. The user also inputs desired hairstyle and color information using an in-app chat interface (e.g., Dialogflow).

[0340] Server Processing

[0341] The server analyzes the received photo and the user's instructions. The natural language processing engine (e.g., NLTK) used analyzes the user's instructions and provides them as prompts to a generative AI model (e.g., StyleGAN2). The generative AI model generates an image based on the user's face photo, with the hairstyle and color modified. This image is temporarily stored in a database (e.g., AWS RDS).

[0342] An example of a prompt sentence would be "Shoulder-length, swept-back bangs, ash gray" and sent to the generation AI.

[0343] Image generation and review

[0344] The image processed by the generative AI model is sent back to the device from the server and displayed in a format that the user can check within "BeautyApp." The user can check the processed image and input instructions for fine-tuning as needed, such as "Make the bangs a little shorter."

[0345] The role of the emotional engine

[0346] While the user is viewing the image, an emotion engine (e.g., Microsoft® Azure® Emotion API) analyzes the user's facial expressions and voice. If the user shows a dissatisfied expression, the emotion engine automatically generates and suggests alternatives. These suggestions are presented to the user through the chat interface.

[0347] Final confirmation and submission

[0348] When the user taps the "OK" button on the image they are finally satisfied with, the device sends the final edited image to the server. The server then sends the final edited image and the results of the user's emotional analysis to the salon. Based on this information, the hairdresser can understand the user's wishes and mood before the treatment. The device then receives a receipt confirmation and notifies the user.

[0349] As a concrete example, a user selects and uploads a selfie and inputs instructions such as "I want shoulder-length hair, swept-back bangs, and an ash gray color." The server analyzes these instructions and generates an edited image using a generative AI model. The user then reviews the image, and an emotion engine analyzes the user's facial expressions and suggests alternatives if the user is dissatisfied. This process is repeated until the final image the user is satisfied with is sent to the hairdresser, and a confirmation of receipt is obtained.

[0350] The present invention allows users to accurately communicate their ideal hairstyle and color and achieve the desired results.

[0351] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0352] Step 1:

[0353] The user installs and launches a dedicated application on their smartphone. The application prompts the user to register and configure the settings. The user then takes a photo of their face or selects an existing photo. The selected or taken photo is the input data.

[0354] Step 2:

[0355] The device converts the selected photo into a data packet and sends it to the server over the Internet. This process includes the following specific operations:

[0356] Loading images

[0357] Image data compression and encryption

[0358] Sending to the server

[0359] The transmitted image data is the output.

[0360] Step 3:

[0361] The server temporarily stores the received image data in an AWS S3 bucket. The stored image data is the input. The server confirms receipt of the image data and returns an output indicating that the data has been saved.

[0362] Step 4:

[0363] Users use the app's chat interface to input their desired hairstyle and color, such as "shoulder-length, swept-back bangs, ash gray." This is the input data.

[0364] Step 5:

[0365] The terminal assembles the user's instructions into a data packet and sends it to the server. The instructions are input. The server then sends the received instructions to the analysis engine and outputs the analysis results.

[0366] Step 6:

[0367] The server analyzes the received instructions using a natural language processing engine (NLTK). The analysis result is the input. The analyzed instructions (prompt sentence) are the output.

[0368] Step 7:

[0369] The server supplies a prompt to a generative AI model (StyleGAN2) to generate an image. The input includes the prompt and a saved facial photo. The generative AI model generates an image with the hairstyle and color applied based on the user's instructions. The generated image is the output.

[0370] Step 8:

[0371] The server temporarily stores the generated image in the AWS database and returns it to the device. Amazon S3 and CloudFront are used for the return. The generated image is the input data. A return completion message is the output.

[0372] Step 9:

[0373] The device displays the generated image to the user. The user checks the displayed image and inputs instructions for fine-tuning as needed. For example, specific instructions such as "Make the bangs a little shorter" are input data. The displayed image is the output.

[0374] Step 10:

[0375] The device sends additional instructions from the user to the server again. The sent instructions are input data. The server then supplies a new prompt to the generation AI in the same way and outputs the recreated image.

[0376] Step 11:

[0377] While the user is viewing the image, the emotion engine (Microsoft Azure's Emotion API) analyzes the user's facial expressions and voice. The analyzed facial expressions and voice are the input data. If the user has a dissatisfied expression, the emotion engine automatically generates and proposes an alternative. This proposal is the output.

[0378] Step 12:

[0379] When the user taps the "OK" button on the image they are finally satisfied with, the device sends the final processed image to the server. The sent image is the input data.

[0380] Step 13:

[0381] The server sends the final processed image and the user's sentiment analysis results to the hair salon. The input data is the hairdresser's contact information. The output is a confirmation message after the transmission is complete.

[0382] Step 14:

[0383] The hairdresser performs the treatment based on the received processed image and the emotion analysis results. Finally, a receipt confirmation from the hairdresser is sent to the user, and the user is notified of the information. The notification data is the output.

[0384] (Application example 2)

[0385] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0386] With conventional virtual try-on systems, it is difficult to accurately simulate the hairstyle and color desired by the user. The lack of functionality to analyze and suggest fine-tuning requests and emotions made it difficult for users to achieve a final result that satisfied them. Furthermore, the inability to send the final edited image to an expert and provide real-time feedback to the user made it difficult to provide optimal service.

[0387] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0388] In this invention, the server includes means for receiving an image taken or selected by the user, means for inputting instructions for the user's desired hairstyle and color, means for processing the image based on the received image in accordance with the user's instructions using generative artificial intelligence, means for displaying the processed image to the user and analyzing the user's facial expressions and voice to recognize emotions, means for receiving the user's instructions for fine-tuning and processing again, and means for saving the final processed image and sending it to an expert. This allows the server to simulate hairstyles and colors while taking the user's emotions into consideration and share the results with the expert in real time, thereby enabling the user to ultimately achieve results that they are satisfied with.

[0389] "Means for receiving images taken or selected by the user" refers to the function that allows the user to upload images they have taken themselves or images they already have to the device, and for the system to receive them.

[0390] "Means for users to input instructions for their desired hairstyle and color" refers to a function that allows users to input their desired hairstyle and color via text or voice and transmit that information to the system.

[0391] "Means for processing an image based on the received image in accordance with the user's instructions using generative artificial intelligence" refers to a function that uses artificial intelligence technology to process the image based on the received image data and the user's instructions.

[0392] "Means of recognizing emotions by displaying a processed image to the user and analyzing the user's facial expressions and voice" refers to a function that displays a processed image to the user, analyzes the user's facial expressions and voice to understand their emotions, and reflects them in the system.

[0393] The "means for receiving a fine adjustment instruction from the user and processing the image again" is a function for receiving a fine adjustment request from the user and processing the image again based on the request.

[0394] The "means for saving the final processed image and sending it to an expert" is a function for saving the final processed image that the user is satisfied with and transferring it to an expert.

[0395] The present invention provides a system that allows a user to simulate hairstyles and colors in an autonomous vehicle and transmit the results to an expert. This system is realized using the following means.

[0396] User Actions

[0397] Using the interface inside the autonomous vehicle, users can take a photo of themselves or select a photo they have already taken from their gallery, then upload the selected photo to a dedicated application, which then sends the photo to a server.

[0398] Server Processing

[0399] The server receives the photo sent from the device and temporarily stores it. It provides an interface for the user to input instructions for the desired hairstyle and color. The user's instructions are sent to the server via the device, and the server analyzes them and provides them to the generative AI model. The generative AI model generates an image with the hairstyle and color applied based on the user's specifications.

[0400] Image generation and display

[0401] The image processed by the generative AI model is sent from the server to the device and displayed for the user to review. As the user reviews the image, the emotion engine analyzes their facial expressions and voice to recognize their emotions. If the user is not satisfied, it will suggest an alternative. Furthermore, it can accept instructions for fine-tuning and process the image again. This process is repeated until the user is finally satisfied.

[0402] Final confirmation and submission

[0403] When the user finally taps the "Decide" button on the image they are satisfied with, the device prepares to send the final edited image to the expert. The server forwards the edited image to the expert's designated contact and obtains the expert's confirmation of receipt. The results of the user's sentiment analysis are also sent to the expert, allowing the expert to understand the user's mood and preferences before providing the service.

[0404] Hardware and software used

[0405] Hardware: Camera, large display, smartphone, head-mounted display

[0406] Software: OpenCV (face detection and image processing), dlib (face detection), Keras (emotion recognition model), generative AI model

[0407] As a concrete example, a user can input instructions through an in-car interface based on a selfie they have uploaded, such as "I want shoulder-length hair, swept-back bangs, and an ash gray color." The server analyzes these instructions and processes the image using a generative AI model. When the user reviews the image, the emotion engine analyzes the user's facial expressions and automatically suggests "making the bangs a little shorter." The user then agrees to the reprocessing, and the server processes the image again using the generative AI model. This process is repeated, and the final image that the user is satisfied with is sent to the expert. The expert then confirms receipt and provides the service, taking into account the results of the emotion analysis.

[0408] Prompt Sentence Examples

[0409] "Long hair, blonde."

[0410] "Shoulder-length hair, swept-back bangs, ash gray."

[0411] As described above, the present invention enables simulation of hairstyles and colors that take into account the user's emotions within an autonomous vehicle, making it possible to smoothly communicate information to experts.

[0412] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0413] Step 1:

[0414] The user launches a dedicated application in the self-driving vehicle and takes a photo of their face or selects a photo they have already saved.

[0415] Input: A photo of your face taken or selected by the user

[0416] Output: Photo data

[0417] How it works: Take a photo using your smartphone or car camera, or select a photo from your gallery and upload it to the dedicated application.

[0418] Step 2:

[0419] The terminal transmits the uploaded photo data to the server.

[0420] Input: Photo data

[0421] Output: Photo data sent to the server

[0422] How it works: Your device uploads photo data to a server via your internet connection.

[0423] Step 3:

[0424] The server temporarily stores the received photo data and provides an interface for the user to input instructions for the desired hairstyle and color.

[0425] Input: Photo data

[0426] Output: Instruction input interface

[0427] How it works: The server stores the photo data in a database and prompts the user for input for the next processing step.

[0428] Step 4:

[0429] The user inputs the desired hairstyle and color and sends the instructions to the server.

[0430] Input: Hair style and color instructions (prompt text)

[0431] Output: User instructions

[0432] How it works: The user enters their desired hairstyle and color via text or voice into a field within the application and submits that information.

[0433] Step 5:

[0434] The server analyzes the user's instructions and supplies them to the generative AI model, which then generates an image with the hairstyle and color applied based on the received photo data and the user's instructions.

[0435] Input: Photo data, user instructions

[0436] Output: Processed image

[0437] How it works: The server analyzes the text of the user's instructions and passes them to a generative AI model to process the image.

[0438] Step 6:

[0439] The server transmits the generated processed image to the terminal and displays it to the user.

[0440] Input: processed image

[0441] Output: Image sent to user device

[0442] How it works: The server sends the generated image data to the user's device, which then displays the image.

[0443] Step 7:

[0444] The user can review the edited image and input instructions for fine-tuning, while the emotion engine analyzes the user's facial expressions and voice to recognize their emotions.

[0445] Input: User emotion data, fine-tuning instructions

[0446] Output: User emotion data and fine-tuning content

[0447] How it works: The application captures the user's facial expressions and voice using a camera and microphone, which the emotion engine analyzes. The user then inputs fine-tuning instructions.

[0448] Step 8:

[0449] The server receives the user's fine-tuning instructions and emotion data and requests the generative AI model to process it again.

[0450] Input: Fine-tuning instructions, user emotion data

[0451] Output: Reprocessed image

[0452] How it works: The server analyzes the emotion data and fine-tuning instructions, then passes new instructions to the generative AI model for reprocessing.

[0453] Step 9:

[0454] When the user finally taps the "Decide" button on the image they are satisfied with, the device prepares the final edited image for sending to the expert and notifies the server.

[0455] Input: Final processed image

[0456] Output: Prepared for sending to a specialist

[0457] How it works: The user taps the "OK" button, and the device notifies the server of the final processed image.

[0458] Step 10:

[0459] The server forwards the final processed image to the expert's designated contact and obtains the expert's acknowledgement of receipt. It also sends the user's emotion data to the expert.

[0460] Input: Final processed image, user emotion data

[0461] Output: Send to expert, acknowledge receipt

[0462] How it works: The server sends the final processed image and emotion data to the expert, receives confirmation data, and provides feedback to the user.

[0463] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0464] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (registered trademark) (Internet search engine).<URL: https: / / openai.com / blog / chatgpt> ), Gemini (registered trademark) (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0465] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0466] [Second embodiment]

[0467] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0468] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0469] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0470] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0471] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0472] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0473] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0474] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0475] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0476] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0477] In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0478] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0479] The present invention is a system that allows users to accurately communicate their ideal hairstyle and color when visiting a beauty salon. This system operates in cooperation with the user, terminal, and server elements. A specific example is shown below.

[0480] User Actions

[0481] The first step in using this system is for the user to install and launch the dedicated application. Next, the user takes a photo of their face or selects a photo they have already taken from their gallery. The selected photo is uploaded within the app. This upload action causes the device to send the user's photo to the server.

[0482] Server Processing

[0483] The server receives the photos sent from the device and temporarily stores them. The user uses a chat interface to input instructions for the desired hairstyle and color. The user's instructions are sent to the server via the device, and the server analyzes them. The analyzed instructions are then provided to a generation AI, which then generates an image with the hairstyle and color modified based on the user's specifications.

[0484] Image generation and display

[0485] The image processed by the generation AI is sent from the server to the device and displayed for the user to review. The user reviews the processed image and inputs fine-tuning instructions as needed. These fine-tuning instructions are sent back to the server, which then requests further processing from the generation AI. Fine-tuning is carried out and a new image is generated. This process is repeated until the user is satisfied.

[0486] Final confirmation and submission

[0487] If the user is satisfied with the final image, they tap the "OK" button. This causes the device to prepare to send the final processed image to the server, which then forwards the image to the designated hairdresser. The hairdresser then confirms receipt, and this confirmation information is sent back to the user via the server.

[0488] Specific examples

[0489] As a concrete example, consider a scenario in which a user launches a dedicated app and selects and uploads a selfie. After this photo is sent to the server, the user uses a chat interface to input instructions such as, "Shoulder-length hair, swept-back bangs, and ash gray color." The server analyzes these instructions and sends a processing request to the generation AI. The generation AI generates an image based on the user's face photo with the hairstyle and color applied, and sends this back to the user. After the user confirms this, they can further input fine-tuning instructions such as, "Make the bangs a little shorter." The server processes the image again using the generation AI, and finally sends the final image they are satisfied with to the hairdresser. As a result, the user can accurately communicate their ideal hairstyle and achieve the desired results.

[0490] The processing flow will be explained below.

[0491] Program processing steps

[0492] 1. User photo submission

[0493] Step 1: Select and upload a photo

[0494] User

[0495] Open the application.

[0496] Tap the "Select Photo" button to choose a photo of your face from your device's gallery or take a new one.

[0497] Tap the "Upload" button to send the photo you selected or took.

[0498] Step 2: Prepare your photos for sending

[0499] Terminal

[0500] Temporarily store photos uploaded by users.

[0501] Sends a request to the server to prepare the photo for sending.

[0502] Step 3: Receive and save photos

[0503] server

[0504] Receives photo data sent from the device.

[0505] Save the photos in a database.

[0506] A confirmation to save the photo will be sent to your device.

[0507] ---

[0508] 2. Specify hairstyle and color

[0509] Step 4: Enter instructions

[0510] User

[0511] Use the app's chat interface to input your desired cut (e.g., "shoulder-length") and color (e.g., "ash gray").

[0512] Tap the "Send" button to send the instructions.

[0513] Step 5: Sending instructions

[0514] Terminal

[0515] Receive and temporarily store instructions from the user.

[0516] The instruction is sent to the server.

[0517] Step 6: Parse instructions and input them into the generative AI

[0518] server

[0519] Analyze the instructions received from the terminal.

[0520] The saved photos and instructions are input into the generative AI system.

[0521] The generative AI generates images with edited hairstyles and colors based on the user's instructions.

[0522] ---

[0523] 3. Instructions for fine-tuning and their reflection

[0524] Step 7: Check the processed image and give instructions for fine adjustments

[0525] User

[0526] The processed image returned from the server is checked within the application.

[0527] If you need to make any tweaks, enter new instructions (e.g., "Make the bangs a little shorter") and tap the "Send" button.

[0528] Step 8: Sending fine-tuning instructions

[0529] Terminal

[0530] Receives new instructions and sends them to the server.

[0531] Step 9: Applying the tweaks

[0532] server

[0533] Re-input the received fine-tuning instructions into the generation AI.

[0534] The generative AI readjusts the image and generates a new one.

[0535] The adjusted image is sent to the device.

[0536] This process is repeated until the user is satisfied.

[0537] ---

[0538] 4. Sending the final image

[0539] Step 10: Check the final image

[0540] User

[0541] Finally, check the image of the hairstyle you are satisfied with in the application.

[0542] Tap the "Decide Now" button.

[0543] Step 11: Prepare your images for delivery

[0544] Terminal

[0545] Prepare the final processed image for sending to the beauty salon.

[0546] Step 12: Send to salon

[0547] server

[0548] Send a message to your salon contact with the final processed image.

[0549] The hairdresser's confirmation of receipt is obtained and the status is fed back to the terminal.

[0550] Save the transmission history in the database.

[0551] Example 1

[0552] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0553] When visiting a hair salon, it is difficult for users to accurately communicate their ideal hairstyle and color. In particular, a communication gap between the user and the hairdresser makes it difficult for users to achieve a satisfactory result. For this reason, a system is needed that allows users to visually confirm their ideal style and accurately communicate it to the hairdresser.

[0554] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0555] In this invention, the server includes a means for receiving an image taken or selected by the user, a means for inputting instructions for the user's desired hairstyle and color, a means for processing the image based on the received image in accordance with the user's instructions using a generative artificial intelligence, a means for displaying the processed image to the user, receiving fine adjustment instructions from the user and processing it again, and a means for saving the final processed image and sending it to the hairdresser. This allows the user to visually confirm their ideal hairstyle and color and accurately communicate them to the hairdresser.

[0556] A "user" is an individual who uses the system to specify their ideal hairstyle and color and actually generate an image.

[0557] "Photo or Selected Image" means an image of a user's face taken or selected from existing images and uploaded to the system.

[0558] "Instructions" refers to the specific input the user makes about the hairstyle and color they desire.

[0559] "Generative AI" refers to an AI technology for processing images based on user instructions, such as generative models.

[0560] "Means for processing images" refers to the functionality for using the generated AI model to edit or modify images taken or selected based on user instructions.

[0561] "Fine-tuning instructions" refer to requests for additional corrections or changes made by the user to the displayed edited image.

[0562] The "final edited image" refers to the final image that the user is satisfied with after making fine adjustments.

[0563] A "hairdresser" is a professional who receives the final edited image sent by the user and performs the actual hairstyle and coloring based on it.

[0564] "Server" refers to the core computer system that receives and stores user images, analyzes instructions, and works with the generating AI.

[0565] The present invention is a system that allows users to accurately communicate their ideal hairstyle and color when visiting a beauty salon. This system achieves its purpose by having the user, terminal, and server work together.

[0566] User Actions

[0567] Users install and launch the dedicated application from the App Store or Google Play. After creating an account and logging in, they can take a selfie using the app's photo-taking function or select an existing photo from their gallery. After selection, the device uploads the photo to the server. For example, if a user takes a selfie and taps the upload button in the app, the device compresses the photo and sends it to the server using a secure protocol.

[0568] Server Processing

[0569] The server receives the image sent by the user and temporarily stores it. The user then inputs their desired hairstyle and color through the chat interface. For example, they might input specific instructions such as "shoulder-length hair, swept-back bangs, and ash gray color." The instructions are sent via the user's device to the server, which analyzes them. The analyzed information is then provided to the generation AI, which then processes the image based on the user's instructions.

[0570] Examples of generative AI that can be used include Stable Diffusion and DALL-E. The server sends appropriate prompts to the generative AI model to generate images. For example, the following prompts are used:

[0571] "Generate an image of the user's face with shoulder-length hair, swept-back bangs, and ash gray hair."

[0572] Image generation and display

[0573] The image processed by the generation AI is sent from the server to the device and displayed for the user to review. At this point, the user can review the generated image and input fine-tuning instructions as needed. For example, they can input fine-tuning instructions such as "Make the bangs a little shorter." The instructions sent back to the server are analyzed, and a new image is generated by sending additional prompts to the generation AI. The image is then sent to the user again, and this process is repeated until the user is satisfied.

[0574] Final confirmation and submission

[0575] When the user finally hits the "OK" button on the image they are satisfied with, the device prepares to send the final processed image to the server, which then forwards the image to a dedicated hairdresser. After the hairdresser confirms receipt, the confirmation information is sent back to the user via the server.

[0576] This system allows users to visually confirm their ideal hairstyle and color and accurately communicate it to the hairdresser, which is expected to result in the user achieving the desired results.

[0577] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0578] Step 1: Initial Setup

[0579] The user installs and launches the dedicated application, creates an account, and logs in.

[0580] Input: Installed apps, user information (email address, password)

[0581] Output: The app launches and the user is logged in.

[0582] Step 2: Upload your photo

[0583] The user takes a photo of their face using the app's photo function or selects an existing photo, and the device uploads the selected photo to the server.

[0584] Input: Captured or selected image

[0585] Output: The image is uploaded to the server.

[0586] Specific operation: For example, when you tap the camera button in the app, the camera starts up, you take a picture, check it, and then tap the upload button. The device compresses the image and sends it to the server using the HTTPS protocol.

[0587] Step 3: User prompts

[0588] Users input their desired hairstyle and color through a chat interface, and the device sends the user's instructions to the server.

[0589] Input: User's hairstyle and color instructions (e.g., "Shoulder-length, swept-back bangs, ash gray color")

[0590] Output: The instructions are sent to the server.

[0591] Specific operation: The user enters text in the chat interface and taps the send button. The device sends the text data to the server.

[0592] Step 4: Instruction analysis and image generation

[0593] The server receives and analyzes the user's instructions. The server sends the analysis results to the generation AI, which issues instructions as prompts. The generation AI generates an image based on the prompts.

[0594] Input: User's instructions (text), user's image data

[0595] Output: processed image

[0596] Specific operation: The server uses a natural language processing engine to analyze the instructions, generate specific prompts such as "Hair length: shoulder length," "Bangs: swept-back," and "Color: ash gray," and sends them to the generation AI. The generation AI then generates an image based on this and sends it back to the server.

[0597] Step 5: Image generation and display

[0598] The image generated by the AI ​​is received and stored by the server, which then sends the image to the device and displays it for the user to view.

[0599] Input: Generated image

[0600] Output: The image is displayed on the user's device.

[0601] Specific operation: The server temporarily saves the generated image file in a database, generates a URL and sends it to the device, where the image is displayed and the user is asked to confirm it.

[0602] Step 6: Fine-tuning instructions

[0603] The user checks the generated image and inputs fine-tuning instructions as needed. The device then sends the fine-tuning instructions to the server.

[0604] Input: User fine-tuning instructions (e.g., "Make my bangs a little shorter")

[0605] Output: Fine-tuning instructions are sent to the server.

[0606] Specific operation: The user enters new instructions in the chat interface and taps the send button. The device sends the text data to the server again.

[0607] Step 7: Reprocessing and display

[0608] The server then sends a new prompt to the AI ​​generator, requesting it to generate a finely tuned image. The AI ​​generator generates a new image, which the server stores and sends to the device.

[0609] Input: New fine-tuning instructions, original image data

[0610] Output: New processed image

[0611] What it does: The server analyzes the new instructions and sends prompts to the generation AI to generate new image data, which is then sent back to the device and displayed to the user.

[0612] Step 8: Final Review and Submission

[0613] If the user is satisfied with the final image, they tap the "OK" button. The device prepares and sends the final image to the server. The server then forwards the final image to the designated hairdresser. The hairdresser confirms receipt, and the server returns the confirmation information to the user.

[0614] Input: Final processed image

[0615] Output: Final image is sent to the hairdresser, confirmation information is fed back to the user.

[0616] Specific operation: When the user taps the "OK" button, the device compresses the final image data and sends it to the server. The server then forwards the image to the hairdresser, receives confirmation from the hairdresser, and sends feedback to the device.

[0617] (Application example 1)

[0618] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0619] There is a problem that it is difficult for hair salon customers to accurately communicate their ideal hairstyle and color to the hairdresser. This problem manifests itself in the form of miscommunication and dissatisfaction with the finished product due to differences in image. In addition, insufficient reservation management and customer service upon arrival are factors that detract from the customer experience at hair salons. To solve these issues, a system is needed that allows users to virtually try on their ideal hairstyle from home or on the go and share their specific image with the hairdresser.

[0620] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0621] In this invention, the server includes a means for receiving images taken or selected by the user, a means for inputting instructions for the user's desired hairstyle and color, a means for processing the received image according to the user's instructions using a generative AI model, a means for displaying the processed image to the user and receiving fine-tuning instructions from the user for further processing, a means for saving the final processed image and sending it to the hairdresser, and a means for managing the user's appointments. This allows users to try out their ideal hairstyle from home or on the go and share their specific image with the hairdresser, facilitating communication and improving satisfaction with the finished product. It also streamlines appointment management and customer service during visits, improving the overall customer experience.

[0622] "User" refers to an individual who uses the system to try out their ideal hairstyle and color and input their instructions.

[0623] "Image receiving means" refers to a device or system that has the function of uploading and receiving photos taken or selected by a user to a server through an application.

[0624] "Instruction input means" refers to a device or system having an interface for a user to input details of the hairstyle or color desired by the user via text or voice.

[0625] "Generative AI model" refers to an algorithm or system that uses artificial intelligence technology to generate or modify images based on information provided by users.

[0626] "Image processing means" refers to a device or system that edits and processes images in accordance with user instructions based on the received image using a generative AI model.

[0627] The term "fine-tuning instruction receiving means" refers to a device or system that has the function of displaying the processed image, receiving additional instructions from the user, and reprocessing the image based on those instructions.

[0628] "Final image storage means" refers to a device or system that stores the final processed image that the user is satisfied with.

[0629] The term "means for transmitting to a hairdresser" refers to a device or system that has the function of transmitting the final processed image to a designated hairdresser.

[0630] "Reservation management means" refers to a device or system that has the function of managing user reservation information and adjusting reservations and schedules.

[0631] "Reprocessing interface" refers to an interface that allows a user to review the generated image and input instructions for reprocessing, if necessary.

[0632] A "prompt" is a text that contains specific instructions for a generative AI model, and refers to an instruction used to improve the accuracy of image generation and processing.

[0633] "Professionals" refers to hairdressers and hairstylists who receive the images and instructions sent by the system and actually perform the hairstyle and coloring.

[0634] This invention is a system that can accurately convey the ideal hairstyle and color when visiting a beauty salon. This system works in cooperation with the user's device, a server, and a generative AI model.

[0635] User Actions

[0636] Users first install the app on their smartphone and launch it. Then, they take a photo of themselves or select a photo they have already taken from their gallery. The selected photo is then uploaded within the app and sent to the server.

[0637] Server Processing

[0638] The server receives the user's photo sent from the device and temporarily stores it. When the user inputs instructions for the desired hairstyle or color using the chat interface, the instructions are sent to the server via the device. The server analyzes the instructions and provides them as prompts to the generative AI model.

[0639] Image generation using generative AI models

[0640] The generative AI model processes the received image based on the user's instructions. For example, if the instruction is "short bob, black hair, swept bangs," the generative AI model will generate an image based on this information.

[0641] Example prompt sentence:

[0642] "Generate an image of a user with a short bob, dark hair, and swept-back bangs."

[0643] Image confirmation and fine adjustment

[0644] The processed image is sent from the server to the device and displayed for the user to review. The user can then confirm that the image is as desired and input fine-tuning instructions as needed. These fine-tuning instructions are sent back to the server, which requests further processing from the generative AI model. This process is repeated until the user is satisfied.

[0645] Final confirmation and submission

[0646] If the user is satisfied with the final image, they tap the "OK" button. The device prepares to send the final processed image to the server, which then forwards the image to the designated hairdresser. The hairdresser confirms receipt, and the confirmation information is sent back to the user via the server.

[0647] Reservation management and store visit support

[0648] In addition, the system is equipped with a function to manage user reservation information, which allows for smooth adjustment of user reservations and streamlines customer service when they visit the store.

[0649] Specific examples

[0650] For example, a user launches a dedicated app and selects and uploads a selfie. After the photo is sent to the server, the user uses a chat interface to input instructions such as, "I want shoulder-length hair, swept-back bangs, and an ash gray color." The server analyzes these instructions and sends a request to the generative AI model to edit the image. The generative AI model generates an image based on the user's face photo with the hairstyle and color applied, and sends this back to the user. After the user confirms the image, the user can again input instructions for fine-tuning, such as, "Make the bangs a little shorter." The server then requests further editing from the generative AI model and generates a new image. This process is repeated until the user is satisfied. The final image is sent to the hairdresser, and appointment information is managed, allowing the user to get the hairstyle they want.

[0651] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0652] Step 1:

[0653] The user installs and launches the dedicated app on their smartphone. Next, they take a photo of their face or select a previously taken photo from the gallery. The selected photo is uploaded within the app. The user's photo is provided as input, and the photo data is sent to the application server. The server then receives the user's image data.

[0654] Step 2:

[0655] The server receives the user's photos sent from the device and temporarily stores them. Once the photo data is received, it prepares to analyze it.

[0656] Step 3:

[0657] The user uses a chat interface to input instructions for the desired hairstyle and color. The instructions are provided in text format as input, and the instructions are sent from the device to a server, which parses the instructions and generates a prompt for the generative AI model.

[0658] Step 4:

[0659] The server sends the generated prompt to the generative AI model. Specifically, the prompt is used to make the generative AI model generate an image with the hairstyle and color applied. The prompt and photo data are provided as input, and the generative AI model outputs an edited image based on these.

[0660] Step 5:

[0661] The processed image generated by the generative AI model is sent from the server to the device, where the server receives the generated image and returns it in a format that can be displayed to the user.

[0662] Step 6:

[0663] The user checks the generated image, and if it is not as desired, inputs instructions for fine-tuning in the chat interface. The fine-tuning instructions are provided as input from the user, and the instruction data is sent from the terminal to the server.

[0664] Step 7:

[0665] The server receives the user's fine-tuning instructions and sends them to the generative AI model as a new prompt. The fine-tuning instructions are provided as input, and the generative AI model outputs a newly processed image.

[0666] Step 8:

[0667] The regenerated processed image is sent from the server to the user's device, where the user can review it again. This process is repeated until the user is satisfied.

[0668] Step 9:

[0669] If the user is satisfied with the final image, he / she presses the "OK" button, providing the final processed image as input, which is then sent to the server.

[0670] Step 10:

[0671] The server stores the final processed image and sends it to the hairdresser. The final processed image is provided as input and forwarded to the hairdresser. The hairdresser confirms receipt, and the confirmation information is fed back to the user through the server.

[0672] Step 11:

[0673] The system manages the user's reservation information and provides hairdressers with information to ensure a smooth customer experience when they visit. The user's reservation information is provided as input and shared with the hairdresser.

[0674] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0675] The present invention is an advanced system that combines a system that accurately conveys the ideal hairstyle and color when visiting a beauty salon with an emotion engine that recognizes the user's emotions. Specific embodiments executed by the user, terminal, and server are described below.

[0676] User Actions

[0677] Users install and launch the app, then take a photo of themselves or select a photo they have already taken from their gallery, and upload the photo within the app. This upload process causes the device to send the photo to the server.

[0678] Server Processing

[0679] The server receives the photos sent from the device and temporarily stores them. Users use a chat interface to input instructions for their desired hairstyle and color. The user's instructions are sent via the device to the server, which analyzes them. The analyzed instructions are then provided to a generation AI, which then generates an image with the hairstyle and color modified based on the user's specifications.

[0680] Image generation and display

[0681] The image processed by the generation AI is sent from the server to the device and displayed for the user to review. The user reviews the processed image and enters instructions for fine-tuning as needed. These instructions are sent back to the server, which then requests further processing from the generation AI. Fine-tuning is carried out and a new image is generated. This process is repeated until the user is satisfied.

[0682] The role of the emotional engine

[0683] As the user reviews the image, the emotion engine analyzes the user's facial expressions and voice to recognize their emotions. For example, if the user looks dissatisfied, the emotion engine automatically suggests an alternative. The emotion analysis results are sent to the server and reflected in the next step of processing.

[0684] Final confirmation and submission

[0685] When the user finally taps the "OK" button on the image they are satisfied with, the device prepares to send the final edited image to the salon. The server forwards this edited image to the salon's designated contact and obtains the hairdresser's confirmation of receipt. The results of the user's sentiment analysis are also sent to the hairdresser, allowing the hairdresser to understand the user's mood and preferences before the treatment. This information is then fed back to the user.

[0686] Specific examples

[0687] As a concrete example, we will show a scene in which a user launches a dedicated application and selects and uploads a selfie. After the photo is sent to the server, the user uses a chat interface to input instructions such as, "I want shoulder-length hair, swept-back bangs, and an ash gray color." The server analyzes these instructions and sends a processing request to the generation AI. The generation AI generates an image based on the user's face photo, applying the hairstyle and color, and sends this back to the user. When the user reviews the image, the emotion engine analyzes the user's facial expression. If the user is not satisfied, it automatically suggests, "Make the bangs a little shorter." The user accepts, and the server uses the generation AI to process the image again. This process is repeated, and finally, the image that the user is satisfied with is sent to the hairdresser. The hairdresser confirms receipt and performs the treatment taking the emotion analysis results into account. As a result, the user can accurately communicate their ideal hairstyle and achieve the desired results.

[0688] The processing flow will be explained below.

[0689] Program processing steps

[0690] 1. User photo submission

[0691] Step 1: Select and upload a photo

[0692] User

[0693] Open the application.

[0694] Tap the "Select Photo" button to choose a photo of your face from your device's gallery or take a new one.

[0695] Tap the "Upload" button to send the photo you selected or took.

[0696] Step 2: Prepare your photos for sending

[0697] Terminal

[0698] Temporarily store photos uploaded by users.

[0699] Sends a request to the server to prepare the photo for sending.

[0700] Step 3: Receive and save photos

[0701] server

[0702] Receives photo data sent from the device.

[0703] Save the photos in a database.

[0704] A confirmation to save the photo will be sent to your device.

[0705] ---

[0706] 2. Specify hairstyle and color

[0707] Step 4: Enter instructions

[0708] User

[0709] Use the app's chat interface to input your desired cut (e.g., "shoulder-length") and color (e.g., "ash gray").

[0710] Tap the "Send" button to send the instructions.

[0711] Step 5: Sending instructions

[0712] Terminal

[0713] Receive and temporarily store instructions from the user.

[0714] The instruction is sent to the server.

[0715] Step 6: Parse instructions and input them into the generative AI

[0716] server

[0717] Analyze the instructions received from the terminal.

[0718] The saved photos and instructions are input into the generative AI system.

[0719] The generative AI generates images with edited hairstyles and colors based on the user's instructions.

[0720] ---

[0721] 3. Instructions for fine-tuning and their reflection

[0722] Step 7: Checking edited images and analyzing sentiment

[0723] User

[0724] The processed image returned from the server is checked within the application.

[0725] During this process, the emotion engine analyzes the user's facial expressions and voice to recognize their emotions.

[0726] Step 8: Emotional Feedback

[0727] server

[0728] The emotion engine receives the user's recognized emotions and presents the next suggestion to the user based on the analysis results.

[0729] For example, if the person looks dissatisfied, the system will make a suggestion such as, "Would you like your bangs cut a little shorter?"

[0730] Step 9: Enter and submit fine-tuning instructions

[0731] User

[0732] Accept the suggestions based on the sentiment analysis results, or enter new fine-tuning instructions (e.g., "Make it a little brighter").

[0733] Send the instructions by tapping the "Send" button.

[0734] Step 10: Analyze and refine the fine-tuning instructions

[0735] server

[0736] Re-input the received fine-tuning instructions into the generation AI.

[0737] The generative AI readjusts the image and generates a new one.

[0738] The adjusted image is sent to the device.

[0739] This process is repeated until the user is satisfied.

[0740] ---

[0741] 4. Sending the final image

[0742] Step 11: Check the final image

[0743] User

[0744] Finally, check the image of the hairstyle you are satisfied with in the application.

[0745] Tap the "Decide Now" button.

[0746] Step 12: Prepare your images for delivery

[0747] Terminal

[0748] Prepare the final processed image for sending to the beauty salon.

[0749] Step 13: Send to salon

[0750] server

[0751] Send a message to your salon contact with the final processed image.

[0752] The hairdresser's confirmation of receipt is obtained and the status is fed back to the terminal.

[0753] Save the transmission history in the database.

[0754] Step 14: Sending sentiment analysis results to hairdressers

[0755] server

[0756] The results of the user's sentiment analysis are also sent to the hairdresser, allowing the hairdresser to understand the user's mood and preferences before the treatment.

[0757] Specific examples

[0758] As a concrete example, consider a scenario in which a user launches a dedicated application, selects and uploads a selfie. After the photo is sent to the server, the user enters instructions in the chat interface, such as "I want shoulder-length hair, swept-back bangs, and an ash gray color," and submits the image. The server analyzes the instructions and generates an edited image using generative AI. When the user reviews the image, the emotion engine suggests, "Maybe the bangs would be better if they were a little shorter." The user accepts the suggestion, and the server again fine-tunes the image using generative AI. The final image that the user is satisfied with is sent to the hairdresser, along with the results of the emotion analysis. This process allows the user to accurately communicate their ideal hairstyle and achieve the desired results.

[0759] Example 2

[0760] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0761] In conventional beauty salons, communication between customers and hairdressers has been hindered by the difficulty of accurately conveying the customer's desired hairstyle and color. Furthermore, if the customer is dissatisfied with the processed image, their feelings are not properly reflected, making it difficult to achieve the desired result. The present invention aims to solve these problems and provide a system that allows customers to accurately convey their desired hairstyle and color and achieve satisfactory results.

[0762] The specification process by the specification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes a means for receiving an image taken or selected by the user, a means for inputting instructions for the user's desired hairstyle and color, a means for processing the image based on the received image in accordance with the user's instructions using a generative artificial intelligence model, a means for displaying the processed image to the user and receiving fine-tuning instructions from the user for further processing, a means for saving the final processed image and sending it to the hairdresser, and a means for recognizing the user's emotions and automatically making suggestions based on the emotions. This allows the customer to accurately communicate their ideal hairstyle and color, ensure that their emotions are properly reflected in the generated image, and ultimately achieve a satisfactory result.

[0763] "User" refers to an individual who uses this system to give instructions about hairstyles and colors and check the results.

[0764] The "server" refers to the infrastructure responsible for processing user images and instructions, processing images using generative AI models, and recognizing user emotions.

[0765] A "generative AI model" is an artificial intelligence algorithm that processes hairstyles and colors based on a user's facial photo and instructions, specifically referring to technologies such as StyleGAN2.

[0766] An "emotion engine" refers to a software component that analyzes a user's facial expressions and voice and recognizes their emotions.

[0767] "Means for receiving images" refers to a function for transmitting an image taken or selected by a user to a server, and for the server to receive the image.

[0768] "Means for inputting instructions" refers to an interface through which a user can input information about the hairstyle and color they desire.

[0769] "Means of processing" refers to the function of processing received images based on user instructions using a generative AI model.

[0770] "Means for receiving fine-tuning instructions and reprocessing" refers to a function for receiving additional processing instructions from the user and reprocessing the image.

[0771] "Means for saving and sending to a hairdresser" refers to the function of saving the final processed image and sending the image to a hairdresser.

[0772] "Means for automatically making emotion-based suggestions" refers to the ability of the emotion engine to recognize the user's emotions and automatically make different suggestions based on the results.

[0773] "Final processed image" refers to the image generated in its final form, reflecting the user's wishes and fine-tuning instructions.

[0774] "Means for providing feedback" refers to features that notify users, such as acknowledgments from their hairdressers, to improve their experience.

[0775] The present invention is an advanced system that combines a system that accurately conveys the ideal hairstyle and color when visiting a beauty salon with an emotion engine that recognizes the user's emotions. Specific embodiments executed by the user, terminal, and server are described below.

[0776] User Actions

[0777] Users install and launch a dedicated application on their smartphone. This application is available for iOS and Android platforms. Users can take a photo of their face using the application's camera function or select a photo they have already taken from their gallery. They can then upload the selected photo within the app. This upload causes the device to send the photo to the server.

[0778] Terminal handling

[0779] The device converts the selected photo into a data packet and sends it over the Internet to a server, which temporarily stores the image in an Amazon Web Services (AWS) S3 bucket. The user also inputs desired hairstyle and color information using an in-app chat interface (e.g., Dialogflow).

[0780] Server Processing

[0781] The server analyzes the received photo and the user's instructions. The natural language processing engine (e.g., NLTK) used analyzes the user's instructions and provides them as prompts to a generative AI model (e.g., StyleGAN2). The generative AI model generates an image based on the user's face photo, with the hairstyle and color modified. This image is temporarily stored in a database (e.g., AWS RDS).

[0782] An example of a prompt sentence would be "Shoulder-length, swept-back bangs, ash gray" and sent to the generation AI.

[0783] Image generation and review

[0784] The image processed by the generative AI model is sent back to the device from the server and displayed in a format that the user can check within "BeautyApp." The user can check the processed image and input instructions for fine-tuning as needed, such as "Make the bangs a little shorter."

[0785] The role of the emotional engine

[0786] While the user is viewing the image, an emotion engine (e.g., Microsoft Azure's Emotion API) analyzes the user's facial expressions and voice. If the user shows a dissatisfied expression, the emotion engine automatically generates and suggests alternatives. These suggestions are presented to the user through the chat interface.

[0787] Final confirmation and submission

[0788] When the user taps the "OK" button on the image they are finally satisfied with, the device sends the final edited image to the server. The server then sends the final edited image and the results of the user's emotional analysis to the salon. Based on this information, the hairdresser can understand the user's wishes and mood before the treatment. The device then receives a receipt confirmation and notifies the user.

[0789] As a concrete example, a user selects and uploads a selfie and inputs instructions such as "I want shoulder-length hair, swept-back bangs, and an ash gray color." The server analyzes these instructions and generates an edited image using a generative AI model. The user then reviews the image, and an emotion engine analyzes the user's facial expressions and suggests alternatives if the user is dissatisfied. This process is repeated until the final image the user is satisfied with is sent to the hairdresser, and a confirmation of receipt is obtained.

[0790] The present invention allows users to accurately communicate their ideal hairstyle and color and achieve the desired results.

[0791] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0792] Step 1:

[0793] The user installs and launches a dedicated application on their smartphone. The application prompts the user to register and configure the settings. The user then takes a photo of their face or selects an existing photo. The selected or taken photo is the input data.

[0794] Step 2:

[0795] The device converts the selected photo into a data packet and sends it to the server over the Internet. This process includes the following specific operations:

[0796] Loading images

[0797] Image data compression and encryption

[0798] Sending to the server

[0799] The transmitted image data is the output.

[0800] Step 3:

[0801] The server temporarily stores the received image data in an AWS S3 bucket. The stored image data is the input. The server confirms receipt of the image data and returns an output indicating that the data has been saved.

[0802] Step 4:

[0803] Users use the app's chat interface to input their desired hairstyle and color, such as "shoulder-length, swept-back bangs, ash gray." This is the input data.

[0804] Step 5:

[0805] The terminal assembles the user's instructions into a data packet and sends it to the server. The instructions are input. The server then sends the received instructions to the analysis engine and outputs the analysis results.

[0806] Step 6:

[0807] The server analyzes the received instructions using a natural language processing engine (NLTK). The analysis result is the input. The analyzed instructions (prompt sentence) are the output.

[0808] Step 7:

[0809] The server supplies a prompt to a generative AI model (StyleGAN2) to generate an image. The input includes the prompt and a saved facial photo. The generative AI model generates an image with the hairstyle and color applied based on the user's instructions. The generated image is the output.

[0810] Step 8:

[0811] The server temporarily stores the generated image in the AWS database and returns it to the device. Amazon S3 and CloudFront are used for the return. The generated image is the input data. A return completion message is the output.

[0812] Step 9:

[0813] The device displays the generated image to the user. The user checks the displayed image and inputs instructions for fine-tuning as needed. For example, specific instructions such as "Make the bangs a little shorter" are input data. The displayed image is the output.

[0814] Step 10:

[0815] The device sends additional instructions from the user to the server again. The sent instructions are input data. The server then supplies a new prompt to the generation AI in the same way and outputs the recreated image.

[0816] Step 11:

[0817] While the user is viewing the image, the emotion engine (Microsoft Azure's Emotion API) analyzes the user's facial expressions and voice. The analyzed facial expressions and voice are the input data. If the user has a dissatisfied expression, the emotion engine automatically generates and proposes an alternative. This proposal is the output.

[0818] Step 12:

[0819] When the user taps the "OK" button on the image they are finally satisfied with, the device sends the final processed image to the server. The sent image is the input data.

[0820] Step 13:

[0821] The server sends the final processed image and the user's sentiment analysis results to the hair salon. The input data is the hairdresser's contact information. The output is a confirmation message after the transmission is complete.

[0822] Step 14:

[0823] The hairdresser performs the treatment based on the received processed image and the emotion analysis results. Finally, a receipt confirmation from the hairdresser is sent to the user, and the user is notified of the information. The notification data is the output.

[0824] (Application example 2)

[0825] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0826] With conventional virtual try-on systems, it is difficult to accurately simulate the hairstyle and color desired by the user. The lack of functionality to analyze and suggest fine-tuning requests and emotions made it difficult for users to achieve a final result that satisfied them. Furthermore, the inability to send the final edited image to an expert and provide real-time feedback to the user made it difficult to provide optimal service.

[0827] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0828] In this invention, the server includes means for receiving an image taken or selected by the user, means for inputting instructions for the user's desired hairstyle and color, means for processing the image based on the received image in accordance with the user's instructions using generative artificial intelligence, means for displaying the processed image to the user and analyzing the user's facial expressions and voice to recognize emotions, means for receiving the user's instructions for fine-tuning and processing again, and means for saving the final processed image and sending it to an expert. This allows the server to simulate hairstyles and colors while taking the user's emotions into consideration and share the results with the expert in real time, thereby enabling the user to ultimately achieve results that they are satisfied with.

[0829] "Means for receiving images taken or selected by the user" refers to the function that allows the user to upload images they have taken themselves or images they already have to the device, and for the system to receive them.

[0830] "Means for users to input instructions for their desired hairstyle and color" refers to a function that allows users to input their desired hairstyle and color via text or voice and transmit that information to the system.

[0831] "Means for processing an image based on the received image in accordance with the user's instructions using generative artificial intelligence" refers to a function that uses artificial intelligence technology to process the image based on the received image data and the user's instructions.

[0832] "Means of recognizing emotions by displaying a processed image to the user and analyzing the user's facial expressions and voice" refers to a function that displays a processed image to the user, analyzes the user's facial expressions and voice to understand their emotions, and reflects them in the system.

[0833] The "means for receiving a fine adjustment instruction from the user and processing the image again" is a function for receiving a fine adjustment request from the user and processing the image again based on the request.

[0834] The "means for saving the final processed image and sending it to an expert" is a function for saving the final processed image that the user is satisfied with and transferring it to an expert.

[0835] The present invention provides a system that allows a user to simulate hairstyles and colors in an autonomous vehicle and transmit the results to an expert. This system is realized using the following means.

[0836] User Actions

[0837] Using the interface inside the autonomous vehicle, users can take a photo of themselves or select a photo they have already taken from their gallery, then upload the selected photo to a dedicated application, which then sends the photo to a server.

[0838] Server Processing

[0839] The server receives the photo sent from the device and temporarily stores it. It provides an interface for the user to input instructions for the desired hairstyle and color. The user's instructions are sent to the server via the device, and the server analyzes them and provides them to the generative AI model. The generative AI model generates an image with the hairstyle and color applied based on the user's specifications.

[0840] Image generation and display

[0841] The image processed by the generative AI model is sent from the server to the device and displayed for the user to review. As the user reviews the image, the emotion engine analyzes their facial expressions and voice to recognize their emotions. If the user is not satisfied, it will suggest an alternative. Furthermore, it can accept instructions for fine-tuning and process the image again. This process is repeated until the user is finally satisfied.

[0842] Final confirmation and submission

[0843] When the user finally taps the "Decide" button on the image they are satisfied with, the device prepares to send the final edited image to the expert. The server forwards the edited image to the expert's designated contact and obtains the expert's confirmation of receipt. The results of the user's sentiment analysis are also sent to the expert, allowing the expert to understand the user's mood and preferences before providing the service.

[0844] Hardware and software used

[0845] Hardware: Camera, large display, smartphone, head-mounted display

[0846] Software: OpenCV (face detection and image processing), dlib (face detection), Keras (emotion recognition model), generative AI model

[0847] As a concrete example, a user can input instructions through an in-car interface based on a selfie they have uploaded, such as "I want shoulder-length hair, swept-back bangs, and an ash gray color." The server analyzes these instructions and processes the image using a generative AI model. When the user reviews the image, the emotion engine analyzes the user's facial expressions and automatically suggests "making the bangs a little shorter." The user then agrees to the reprocessing, and the server processes the image again using the generative AI model. This process is repeated, and the final image that the user is satisfied with is sent to the expert. The expert then confirms receipt and provides the service, taking into account the results of the emotion analysis.

[0848] Prompt Sentence Examples

[0849] "Long hair, blonde."

[0850] "Shoulder-length hair, swept-back bangs, ash gray."

[0851] As described above, the present invention enables simulation of hairstyles and colors that take into account the user's emotions within an autonomous vehicle, making it possible to smoothly communicate information to experts.

[0852] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0853] Step 1:

[0854] The user launches a dedicated application in the self-driving vehicle and takes a photo of their face or selects a photo they have already saved.

[0855] Input: A photo of your face taken or selected by the user

[0856] Output: Photo data

[0857] How it works: Take a photo using your smartphone or car camera, or select a photo from your gallery and upload it to the dedicated application.

[0858] Step 2:

[0859] The terminal transmits the uploaded photo data to the server.

[0860] Input: Photo data

[0861] Output: Photo data sent to the server

[0862] How it works: Your device uploads photo data to a server via your internet connection.

[0863] Step 3:

[0864] The server temporarily stores the received photo data and provides an interface for the user to input instructions for the desired hairstyle and color.

[0865] Input: Photo data

[0866] Output: Instruction input interface

[0867] How it works: The server stores the photo data in a database and prompts the user for input for the next processing step.

[0868] Step 4:

[0869] The user inputs the desired hairstyle and color and sends the instructions to the server.

[0870] Input: Hair style and color instructions (prompt text)

[0871] Output: User instructions

[0872] How it works: The user enters their desired hairstyle and color via text or voice into a field within the application and submits that information.

[0873] Step 5:

[0874] The server analyzes the user's instructions and supplies them to the generative AI model, which then generates an image with the hairstyle and color applied based on the received photo data and the user's instructions.

[0875] Input: Photo data, user instructions

[0876] Output: Processed image

[0877] How it works: The server analyzes the text of the user's instructions and passes them to a generative AI model to process the image.

[0878] Step 6:

[0879] The server transmits the generated processed image to the terminal and displays it to the user.

[0880] Input: processed image

[0881] Output: Image sent to user device

[0882] How it works: The server sends the generated image data to the user's device, which then displays the image.

[0883] Step 7:

[0884] The user can review the edited image and input instructions for fine-tuning, while the emotion engine analyzes the user's facial expressions and voice to recognize their emotions.

[0885] Input: User emotion data, fine-tuning instructions

[0886] Output: User emotion data and fine-tuning content

[0887] How it works: The application captures the user's facial expressions and voice using a camera and microphone, which the emotion engine analyzes. The user then inputs fine-tuning instructions.

[0888] Step 8:

[0889] The server receives the user's fine-tuning instructions and emotion data and requests the generative AI model to process it again.

[0890] Input: Fine-tuning instructions, user emotion data

[0891] Output: Reprocessed image

[0892] How it works: The server analyzes the emotion data and fine-tuning instructions, then passes new instructions to the generative AI model for reprocessing.

[0893] Step 9:

[0894] When the user finally taps the "Decide" button on the image they are satisfied with, the device prepares the final edited image for sending to the expert and notifies the server.

[0895] Input: Final processed image

[0896] Output: Prepared for sending to a specialist

[0897] How it works: The user taps the "OK" button, and the device notifies the server of the final processed image.

[0898] Step 10:

[0899] The server forwards the final processed image to the expert's designated contact and obtains the expert's acknowledgement of receipt. It also sends the user's emotion data to the expert.

[0900] Input: Final processed image, user emotion data

[0901] Output: Send to expert, acknowledge receipt

[0902] How it works: The server sends the final processed image and emotion data to the expert, receives confirmation data, and provides feedback to the user.

[0903] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0904] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0905] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[0906] [Third embodiment]

[0907] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[0908] 5, the data processing system 310 includes the data processing device 12 and a headset type terminal 314. An example of the data processing device 12 is a server.

[0909] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0910] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[0911] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0912] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0913] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0914] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0915] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0916] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0917] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0918] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[0919] The present invention is a system that allows users to accurately communicate their ideal hairstyle and color when visiting a beauty salon. This system operates in cooperation with the user, terminal, and server elements. A specific example is shown below.

[0920] User Actions

[0921] The first step in using this system is for the user to install and launch the dedicated application. Next, the user takes a photo of their face or selects a photo they have already taken from their gallery. The selected photo is uploaded within the app. This upload action causes the device to send the user's photo to the server.

[0922] Server Processing

[0923] The server receives the photos sent from the device and temporarily stores them. The user uses a chat interface to input instructions for the desired hairstyle and color. The user's instructions are sent to the server via the device, and the server analyzes them. The analyzed instructions are then provided to a generation AI, which then generates an image with the hairstyle and color modified based on the user's specifications.

[0924] Image generation and display

[0925] The image processed by the generation AI is sent from the server to the device and displayed for the user to review. The user reviews the processed image and inputs fine-tuning instructions as needed. These fine-tuning instructions are sent back to the server, which then requests further processing from the generation AI. Fine-tuning is carried out and a new image is generated. This process is repeated until the user is satisfied.

[0926] Final confirmation and submission

[0927] If the user is satisfied with the final image, they tap the "OK" button. This causes the device to prepare to send the final processed image to the server, which then forwards the image to the designated hairdresser. The hairdresser then confirms receipt, and this confirmation information is sent back to the user via the server.

[0928] Specific examples

[0929] As a concrete example, consider a scenario in which a user launches a dedicated app and selects and uploads a selfie. After this photo is sent to the server, the user uses a chat interface to input instructions such as, "Shoulder-length hair, swept-back bangs, and ash gray color." The server analyzes these instructions and sends a processing request to the generation AI. The generation AI generates an image based on the user's face photo with the hairstyle and color applied, and sends this back to the user. After the user confirms this, they can further input fine-tuning instructions such as, "Make the bangs a little shorter." The server processes the image again using the generation AI, and finally sends the final image they are satisfied with to the hairdresser. As a result, the user can accurately communicate their ideal hairstyle and achieve the desired results.

[0930] The processing flow will be explained below.

[0931] Program processing steps

[0932] 1. User photo submission

[0933] Step 1: Select and upload a photo

[0934] User

[0935] Open the application.

[0936] Tap the "Select Photo" button to choose a photo of your face from your device's gallery or take a new one.

[0937] Tap the "Upload" button to send the photo you selected or took.

[0938] Step 2: Prepare your photos for sending

[0939] Terminal

[0940] Temporarily store photos uploaded by users.

[0941] Sends a request to the server to prepare the photo for sending.

[0942] Step 3: Receive and save photos

[0943] server

[0944] Receives photo data sent from the device.

[0945] Save the photos in a database.

[0946] A confirmation to save the photo will be sent to your device.

[0947] ---

[0948] 2. Specify hairstyle and color

[0949] Step 4: Enter instructions

[0950] User

[0951] Use the app's chat interface to input your desired cut (e.g., "shoulder-length") and color (e.g., "ash gray").

[0952] Tap the "Send" button to send the instructions.

[0953] Step 5: Sending instructions

[0954] Terminal

[0955] Receive and temporarily store instructions from the user.

[0956] The instruction is sent to the server.

[0957] Step 6: Parse instructions and input them into the generative AI

[0958] server

[0959] Analyze the instructions received from the terminal.

[0960] The saved photos and instructions are input into the generative AI system.

[0961] The generative AI generates images with edited hairstyles and colors based on the user's instructions.

[0962] ---

[0963] 3. Instructions for fine-tuning and their reflection

[0964] Step 7: Check the processed image and give instructions for fine adjustments

[0965] User

[0966] The processed image returned from the server is checked within the application.

[0967] If you need to make any tweaks, enter new instructions (e.g., "Make the bangs a little shorter") and tap the "Send" button.

[0968] Step 8: Sending fine-tuning instructions

[0969] Terminal

[0970] Receives new instructions and sends them to the server.

[0971] Step 9: Applying the tweaks

[0972] server

[0973] Re-input the received fine-tuning instructions into the generation AI.

[0974] The generative AI readjusts the image and generates a new one.

[0975] The adjusted image is sent to the device.

[0976] This process is repeated until the user is satisfied.

[0977] ---

[0978] 4. Sending the final image

[0979] Step 10: Check the final image

[0980] User

[0981] Finally, check the image of the hairstyle you are satisfied with in the application.

[0982] Tap the "Decide Now" button.

[0983] Step 11: Prepare your images for delivery

[0984] Terminal

[0985] Prepare the final processed image for sending to the beauty salon.

[0986] Step 12: Send to salon

[0987] server

[0988] Send a message to your salon contact with the final processed image.

[0989] The hairdresser's confirmation of receipt is obtained and the status is fed back to the terminal.

[0990] Save the transmission history in the database.

[0991] Example 1

[0992] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[0993] When visiting a hair salon, it is difficult for users to accurately communicate their ideal hairstyle and color. In particular, a communication gap between the user and the hairdresser makes it difficult for users to achieve a satisfactory result. For this reason, a system is needed that allows users to visually confirm their ideal style and accurately communicate it to the hairdresser.

[0994] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0995] In this invention, the server includes a means for receiving an image taken or selected by the user, a means for inputting instructions for the user's desired hairstyle and color, a means for processing the image based on the received image in accordance with the user's instructions using a generative artificial intelligence, a means for displaying the processed image to the user, receiving fine adjustment instructions from the user and processing it again, and a means for saving the final processed image and sending it to the hairdresser. This allows the user to visually confirm their ideal hairstyle and color and accurately communicate them to the hairdresser.

[0996] A "user" is an individual who uses the system to specify their ideal hairstyle and color and actually generate an image.

[0997] "Photo or Selected Image" means an image of a user's face taken or selected from existing images and uploaded to the system.

[0998] "Instructions" refers to the specific input the user makes about the hairstyle and color they desire.

[0999] "Generative AI" refers to an AI technology for processing images based on user instructions, such as generative models.

[1000] "Means for processing images" refers to the functionality for using the generated AI model to edit or modify images taken or selected based on user instructions.

[1001] "Fine-tuning instructions" refer to requests for additional corrections or changes made by the user to the displayed edited image.

[1002] The "final edited image" refers to the final image that the user is satisfied with after making fine adjustments.

[1003] A "hairdresser" is a professional who receives the final edited image sent by the user and performs the actual hairstyle and coloring based on it.

[1004] "Server" refers to the core computer system that receives and stores user images, analyzes instructions, and works with the generating AI.

[1005] The present invention is a system that allows users to accurately communicate their ideal hairstyle and color when visiting a beauty salon. This system achieves its purpose by having the user, terminal, and server work together.

[1006] User Actions

[1007] Users install and launch the app from the App Store or Google Play. After creating an account and logging in, they can either take a selfie using the app's photo feature or select an existing photo from their gallery. After selecting, the device uploads the photo to the server. For example, if a user takes a selfie and taps the upload button in the app, the device compresses the photo and sends it to the server using a secure protocol.

[1008] Server Processing

[1009] The server receives the image sent by the user and temporarily stores it. The user then inputs their desired hairstyle and color through the chat interface. For example, they might input specific instructions such as "shoulder-length hair, swept-back bangs, and ash gray color." The instructions are sent via the user's device to the server, which analyzes them. The analyzed information is then provided to the generation AI, which then processes the image based on the user's instructions.

[1010] Examples of generative AI that can be used include Stable Diffusion and DALL-E. The server sends appropriate prompts to the generative AI model to generate images. For example, the following prompts are used:

[1011] "Generate an image of the user's face with shoulder-length hair, swept-back bangs, and ash gray hair."

[1012] Image generation and display

[1013] The image processed by the generation AI is sent from the server to the device and displayed for the user to review. At this point, the user can review the generated image and input fine-tuning instructions as needed. For example, they can input fine-tuning instructions such as "Make the bangs a little shorter." The instructions sent back to the server are analyzed, and a new image is generated by sending additional prompts to the generation AI. The image is then sent to the user again, and this process is repeated until the user is satisfied.

[1014] Final confirmation and submission

[1015] When the user finally hits the "OK" button on the image they are satisfied with, the device prepares to send the final processed image to the server, which then forwards the image to a dedicated hairdresser. After the hairdresser confirms receipt, the confirmation information is sent back to the user via the server.

[1016] This system allows users to visually confirm their ideal hairstyle and color and accurately communicate it to the hairdresser, which is expected to result in the user achieving the desired results.

[1017] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1018] Step 1: Initial Setup

[1019] The user installs and launches the dedicated application, creates an account, and logs in.

[1020] Input: Installed apps, user information (email address, password)

[1021] Output: The app launches and the user is logged in.

[1022] Step 2: Upload your photo

[1023] The user takes a photo of their face using the app's photo function or selects an existing photo, and the device uploads the selected photo to the server.

[1024] Input: Captured or selected image

[1025] Output: The image is uploaded to the server.

[1026] Specific operation: For example, when you tap the camera button in the app, the camera starts up, you take a picture, check it, and then tap the upload button. The device compresses the image and sends it to the server using the HTTPS protocol.

[1027] Step 3: User prompts

[1028] Users input their desired hairstyle and color through a chat interface, and the device sends the user's instructions to the server.

[1029] Input: User's hairstyle and color instructions (e.g., "Shoulder-length, swept-back bangs, ash gray color")

[1030] Output: The instructions are sent to the server.

[1031] Specific operation: The user enters text in the chat interface and taps the send button. The device sends the text data to the server.

[1032] Step 4: Instruction analysis and image generation

[1033] The server receives and analyzes the user's instructions. The server sends the analysis results to the generation AI, which issues instructions as prompts. The generation AI generates an image based on the prompts.

[1034] Input: User's instructions (text), user's image data

[1035] Output: processed image

[1036] Specific operation: The server uses a natural language processing engine to analyze the instructions, generate specific prompts such as "Hair length: shoulder length," "Bangs: swept-back," and "Color: ash gray," and sends them to the generation AI. The generation AI then generates an image based on this and sends it back to the server.

[1037] Step 5: Image generation and display

[1038] The image generated by the AI ​​is received and stored by the server, which then sends the image to the device and displays it for the user to view.

[1039] Input: Generated image

[1040] Output: The image is displayed on the user's device.

[1041] Specific operation: The server temporarily saves the generated image file in a database, generates a URL and sends it to the device, where the image is displayed and the user is asked to confirm it.

[1042] Step 6: Fine-tuning instructions

[1043] The user checks the generated image and inputs fine-tuning instructions as needed. The device then sends the fine-tuning instructions to the server.

[1044] Input: User fine-tuning instructions (e.g., "Make my bangs a little shorter")

[1045] Output: Fine-tuning instructions are sent to the server.

[1046] Specific operation: The user enters new instructions in the chat interface and taps the send button. The device sends the text data to the server again.

[1047] Step 7: Reprocessing and display

[1048] The server then sends a new prompt to the AI ​​generator, requesting it to generate a finely tuned image. The AI ​​generator generates a new image, which the server stores and sends to the device.

[1049] Input: New fine-tuning instructions, original image data

[1050] Output: New processed image

[1051] What it does: The server analyzes the new instructions and sends prompts to the generation AI to generate new image data, which is then sent back to the device and displayed to the user.

[1052] Step 8: Final Review and Submission

[1053] If the user is satisfied with the final image, they tap the "OK" button. The device prepares and sends the final image to the server. The server then forwards the final image to the designated hairdresser. The hairdresser confirms receipt, and the server returns the confirmation information to the user.

[1054] Input: Final processed image

[1055] Output: Final image is sent to the hairdresser, confirmation information is fed back to the user.

[1056] Specific operation: When the user taps the "OK" button, the device compresses the final image data and sends it to the server. The server then forwards the image to the hairdresser, receives confirmation from the hairdresser, and sends feedback to the device.

[1057] (Application example 1)

[1058] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1059] There is a problem that it is difficult for hair salon customers to accurately communicate their ideal hairstyle and color to the hairdresser. This problem manifests itself in the form of miscommunication and dissatisfaction with the finished product due to differences in image. In addition, insufficient reservation management and customer service upon arrival are factors that detract from the customer experience at hair salons. To solve these issues, a system is needed that allows users to virtually try on their ideal hairstyle from home or on the go and share their specific image with the hairdresser.

[1060] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1061] In this invention, the server includes a means for receiving images taken or selected by the user, a means for inputting instructions for the user's desired hairstyle and color, a means for processing the received image according to the user's instructions using a generative AI model, a means for displaying the processed image to the user and receiving fine-tuning instructions from the user for further processing, a means for saving the final processed image and sending it to the hairdresser, and a means for managing the user's appointments. This allows users to try out their ideal hairstyle from home or on the go and share their specific image with the hairdresser, facilitating communication and improving satisfaction with the finished product. It also streamlines appointment management and customer service during visits, improving the overall customer experience.

[1062] "User" refers to an individual who uses the system to try out their ideal hairstyle and color and input their instructions.

[1063] "Image receiving means" refers to a device or system that has the function of uploading and receiving photos taken or selected by a user to a server through an application.

[1064] "Instruction input means" refers to a device or system having an interface for a user to input details of the hairstyle or color desired by the user via text or voice.

[1065] "Generative AI model" refers to an algorithm or system that uses artificial intelligence technology to generate or modify images based on information provided by users.

[1066] "Image processing means" refers to a device or system that edits and processes images in accordance with user instructions based on the received image using a generative AI model.

[1067] The term "fine-tuning instruction receiving means" refers to a device or system that has the function of displaying the processed image, receiving additional instructions from the user, and reprocessing the image based on those instructions.

[1068] "Final image storage means" refers to a device or system that stores the final processed image that the user is satisfied with.

[1069] The term "means for transmitting to a hairdresser" refers to a device or system that has the function of transmitting the final processed image to a designated hairdresser.

[1070] "Reservation management means" refers to a device or system that has the function of managing user reservation information and adjusting reservations and schedules.

[1071] "Reprocessing interface" refers to an interface that allows a user to review the generated image and input instructions for reprocessing, if necessary.

[1072] A "prompt" is a text that contains specific instructions for a generative AI model, and refers to an instruction used to improve the accuracy of image generation and processing.

[1073] "Professionals" refers to hairdressers and hairstylists who receive the images and instructions sent by the system and actually perform the hairstyle and coloring.

[1074] This invention is a system that can accurately convey the ideal hairstyle and color when visiting a beauty salon. This system works in cooperation with the user's device, a server, and a generative AI model.

[1075] User Actions

[1076] Users first install the app on their smartphone and launch it. Then, they take a photo of themselves or select a photo they have already taken from their gallery. The selected photo is then uploaded within the app and sent to the server.

[1077] Server Processing

[1078] The server receives the user's photo sent from the device and temporarily stores it. When the user inputs instructions for the desired hairstyle or color using the chat interface, the instructions are sent to the server via the device. The server analyzes the instructions and provides them as prompts to the generative AI model.

[1079] Image generation using generative AI models

[1080] The generative AI model processes the received image based on the user's instructions. For example, if the instruction is "short bob, black hair, swept bangs," the generative AI model will generate an image based on this information.

[1081] Example prompt sentence:

[1082] "Generate an image of a user with a short bob, dark hair, and swept-back bangs."

[1083] Image confirmation and fine adjustment

[1084] The processed image is sent from the server to the device and displayed for the user to review. The user can then confirm that the image is as desired and input fine-tuning instructions as needed. These fine-tuning instructions are sent back to the server, which requests further processing from the generative AI model. This process is repeated until the user is satisfied.

[1085] Final confirmation and submission

[1086] If the user is satisfied with the final image, they tap the "OK" button. The device prepares to send the final processed image to the server, which then forwards the image to the designated hairdresser. The hairdresser confirms receipt, and the confirmation information is sent back to the user via the server.

[1087] Reservation management and store visit support

[1088] In addition, the system is equipped with a function to manage user reservation information, which allows for smooth adjustment of user reservations and streamlines customer service when they visit the store.

[1089] Specific examples

[1090] For example, a user launches a dedicated app and selects and uploads a selfie. After the photo is sent to the server, the user uses a chat interface to input instructions such as, "I want shoulder-length hair, swept-back bangs, and an ash gray color." The server analyzes these instructions and sends a request to the generative AI model to edit the image. The generative AI model generates an image based on the user's face photo with the hairstyle and color applied, and sends this back to the user. After the user confirms the image, the user can again input instructions for fine-tuning, such as, "Make the bangs a little shorter." The server then requests further editing from the generative AI model and generates a new image. This process is repeated until the user is satisfied. The final image is sent to the hairdresser, and appointment information is managed, allowing the user to get the hairstyle they want.

[1091] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1092] Step 1:

[1093] The user installs and launches the dedicated app on their smartphone. Next, they take a photo of their face or select a previously taken photo from the gallery. The selected photo is uploaded within the app. The user's photo is provided as input, and the photo data is sent to the application server. The server then receives the user's image data.

[1094] Step 2:

[1095] The server receives the user's photos sent from the device and temporarily stores them. Once the photo data is received, it prepares to analyze it.

[1096] Step 3:

[1097] The user uses a chat interface to input instructions for the desired hairstyle and color. The instructions are provided in text format as input, and the instructions are sent from the device to a server, which parses the instructions and generates a prompt for the generative AI model.

[1098] Step 4:

[1099] The server sends the generated prompt to the generative AI model. Specifically, the prompt is used to make the generative AI model generate an image with the hairstyle and color applied. The prompt and photo data are provided as input, and the generative AI model outputs an edited image based on these.

[1100] Step 5:

[1101] The processed image generated by the generative AI model is sent from the server to the device, where the server receives the generated image and returns it in a format that can be displayed to the user.

[1102] Step 6:

[1103] The user checks the generated image, and if it is not as desired, inputs instructions for fine-tuning in the chat interface. The fine-tuning instructions are provided as input from the user, and the instruction data is sent from the terminal to the server.

[1104] Step 7:

[1105] The server receives the user's fine-tuning instructions and sends them to the generative AI model as a new prompt. The fine-tuning instructions are provided as input, and the generative AI model outputs a newly processed image.

[1106] Step 8:

[1107] The regenerated processed image is sent from the server to the user's device, where the user can review it again. This process is repeated until the user is satisfied.

[1108] Step 9:

[1109] If the user is satisfied with the final image, he / she presses the "OK" button, providing the final processed image as input, which is then sent to the server.

[1110] Step 10:

[1111] The server stores the final processed image and sends it to the hairdresser. The final processed image is provided as input and forwarded to the hairdresser. The hairdresser confirms receipt, and the confirmation information is fed back to the user through the server.

[1112] Step 11:

[1113] The system manages the user's reservation information and provides hairdressers with information to ensure a smooth customer experience when they visit. The user's reservation information is provided as input and shared with the hairdresser.

[1114] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1115] The present invention is an advanced system that combines a system that accurately conveys the ideal hairstyle and color when visiting a beauty salon with an emotion engine that recognizes the user's emotions. Specific embodiments executed by the user, terminal, and server are described below.

[1116] User Actions

[1117] Users install and launch the app, then take a photo of themselves or select a photo they have already taken from their gallery, and upload the photo within the app. This upload process causes the device to send the photo to the server.

[1118] Server Processing

[1119] The server receives the photos sent from the device and temporarily stores them. Users use a chat interface to input instructions for their desired hairstyle and color. The user's instructions are sent via the device to the server, which analyzes them. The analyzed instructions are then provided to a generation AI, which then generates an image with the hairstyle and color modified based on the user's specifications.

[1120] Image generation and display

[1121] The image processed by the generation AI is sent from the server to the device and displayed for the user to review. The user reviews the processed image and enters instructions for fine-tuning as needed. These instructions are sent back to the server, which then requests further processing from the generation AI. Fine-tuning is carried out and a new image is generated. This process is repeated until the user is satisfied.

[1122] The role of the emotional engine

[1123] As the user reviews the image, the emotion engine analyzes the user's facial expressions and voice to recognize their emotions. For example, if the user looks dissatisfied, the emotion engine automatically suggests an alternative. The emotion analysis results are sent to the server and reflected in the next step of processing.

[1124] Final confirmation and submission

[1125] When the user finally taps the "OK" button on the image they are satisfied with, the device prepares to send the final edited image to the salon. The server forwards this edited image to the salon's designated contact and obtains the hairdresser's confirmation of receipt. The results of the user's sentiment analysis are also sent to the hairdresser, allowing the hairdresser to understand the user's mood and preferences before the treatment. This information is then fed back to the user.

[1126] Specific examples

[1127] As a concrete example, we will show a scene in which a user launches a dedicated application and selects and uploads a selfie. After the photo is sent to the server, the user uses a chat interface to input instructions such as, "I want shoulder-length hair, swept-back bangs, and an ash gray color." The server analyzes these instructions and sends a processing request to the generation AI. The generation AI generates an image based on the user's face photo, applying the hairstyle and color, and sends this back to the user. When the user reviews the image, the emotion engine analyzes the user's facial expression. If the user is not satisfied, it automatically suggests, "Make the bangs a little shorter." The user accepts, and the server uses the generation AI to process the image again. This process is repeated, and finally, the image that the user is satisfied with is sent to the hairdresser. The hairdresser confirms receipt and performs the treatment taking the emotion analysis results into account. As a result, the user can accurately communicate their ideal hairstyle and achieve the desired results.

[1128] The processing flow will be explained below.

[1129] Program processing steps

[1130] 1. User photo submission

[1131] Step 1: Select and upload a photo

[1132] User

[1133] Open the application.

[1134] Tap the "Select Photo" button to choose a photo of your face from your device's gallery or take a new one.

[1135] Tap the "Upload" button to send the photo you selected or took.

[1136] Step 2: Prepare your photos for sending

[1137] Terminal

[1138] Temporarily store photos uploaded by users.

[1139] Sends a request to the server to prepare the photo for sending.

[1140] Step 3: Receive and save photos

[1141] server

[1142] Receives photo data sent from the device.

[1143] Save the photos in a database.

[1144] A confirmation to save the photo will be sent to your device.

[1145] ---

[1146] 2. Specify hairstyle and color

[1147] Step 4: Enter instructions

[1148] User

[1149] Use the app's chat interface to input your desired cut (e.g., "shoulder-length") and color (e.g., "ash gray").

[1150] Tap the "Send" button to send the instructions.

[1151] Step 5: Sending instructions

[1152] Terminal

[1153] Receive and temporarily store instructions from the user.

[1154] The instruction is sent to the server.

[1155] Step 6: Parse instructions and input them into the generative AI

[1156] server

[1157] Analyze the instructions received from the terminal.

[1158] The saved photos and instructions are input into the generative AI system.

[1159] The generative AI generates images with edited hairstyles and colors based on the user's instructions.

[1160] ---

[1161] 3. Instructions for fine-tuning and their reflection

[1162] Step 7: Checking edited images and analyzing sentiment

[1163] User

[1164] The processed image returned from the server is checked within the application.

[1165] During this process, the emotion engine analyzes the user's facial expressions and voice to recognize their emotions.

[1166] Step 8: Emotional Feedback

[1167] server

[1168] The emotion engine receives the user's recognized emotions and presents the next suggestion to the user based on the analysis results.

[1169] For example, if the person looks dissatisfied, the system will make a suggestion such as, "Would you like your bangs cut a little shorter?"

[1170] Step 9: Enter and submit fine-tuning instructions

[1171] User

[1172] Accept the suggestions based on the sentiment analysis results, or enter new fine-tuning instructions (e.g., "Make it a little brighter").

[1173] Send the instructions by tapping the "Send" button.

[1174] Step 10: Analyze and refine the fine-tuning instructions

[1175] server

[1176] Re-input the received fine-tuning instructions into the generation AI.

[1177] The generative AI readjusts the image and generates a new one.

[1178] The adjusted image is sent to the device.

[1179] This process is repeated until the user is satisfied.

[1180] ---

[1181] 4. Sending the final image

[1182] Step 11: Check the final image

[1183] User

[1184] Finally, check the image of the hairstyle you are satisfied with in the application.

[1185] Tap the "Decide Now" button.

[1186] Step 12: Prepare your images for delivery

[1187] Terminal

[1188] Prepare the final processed image for sending to the beauty salon.

[1189] Step 13: Send to salon

[1190] server

[1191] Send a message to your salon contact with the final processed image.

[1192] The hairdresser's confirmation of receipt is obtained and the status is fed back to the terminal.

[1193] Save the transmission history in the database.

[1194] Step 14: Sending sentiment analysis results to hairdressers

[1195] server

[1196] The results of the user's sentiment analysis are also sent to the hairdresser, allowing the hairdresser to understand the user's mood and preferences before the treatment.

[1197] Specific examples

[1198] As a concrete example, consider a scenario in which a user launches a dedicated application, selects and uploads a selfie. After the photo is sent to the server, the user enters instructions in the chat interface, such as "I want shoulder-length hair, swept-back bangs, and an ash gray color," and submits the image. The server analyzes the instructions and generates an edited image using generative AI. When the user reviews the image, the emotion engine suggests, "Maybe the bangs would be better if they were a little shorter." The user accepts the suggestion, and the server again fine-tunes the image using generative AI. The final image that the user is satisfied with is sent to the hairdresser, along with the results of the emotion analysis. This process allows the user to accurately communicate their ideal hairstyle and achieve the desired results.

[1199] Example 2

[1200] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1201] In conventional beauty salons, communication between customers and hairdressers has been hindered by the difficulty of accurately conveying the customer's desired hairstyle and color. Furthermore, if the customer is dissatisfied with the processed image, their feelings are not properly reflected, making it difficult to achieve the desired result. The present invention aims to solve these problems and provide a system that allows customers to accurately convey their desired hairstyle and color and achieve satisfactory results.

[1202] The specification process by the specification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes a means for receiving an image taken or selected by the user, a means for inputting instructions for the user's desired hairstyle and color, a means for processing the image based on the received image in accordance with the user's instructions using a generative artificial intelligence model, a means for displaying the processed image to the user and receiving fine-tuning instructions from the user for further processing, a means for saving the final processed image and sending it to the hairdresser, and a means for recognizing the user's emotions and automatically making suggestions based on the emotions. This allows the customer to accurately communicate their ideal hairstyle and color, ensure that their emotions are properly reflected in the generated image, and ultimately achieve a satisfactory result.

[1203] "User" refers to an individual who uses this system to give instructions about hairstyles and colors and check the results.

[1204] The "server" refers to the infrastructure responsible for processing user images and instructions, processing images using generative AI models, and recognizing user emotions.

[1205] A "generative AI model" is an artificial intelligence algorithm that processes hairstyles and colors based on a user's facial photo and instructions, specifically referring to technologies such as StyleGAN2.

[1206] An "emotion engine" refers to a software component that analyzes a user's facial expressions and voice and recognizes their emotions.

[1207] "Means for receiving images" refers to a function for transmitting an image taken or selected by a user to a server, and for the server to receive the image.

[1208] "Means for inputting instructions" refers to an interface through which a user can input information about the hairstyle and color they desire.

[1209] "Means of processing" refers to the function of processing received images based on user instructions using a generative AI model.

[1210] "Means for receiving fine-tuning instructions and reprocessing" refers to a function for receiving additional processing instructions from the user and reprocessing the image.

[1211] "Means for saving and sending to a hairdresser" refers to the function of saving the final processed image and sending the image to a hairdresser.

[1212] "Means for automatically making emotion-based suggestions" refers to the ability of the emotion engine to recognize the user's emotions and automatically make different suggestions based on the results.

[1213] "Final processed image" refers to the image generated in its final form, reflecting the user's wishes and fine-tuning instructions.

[1214] "Means for providing feedback" refers to features that notify users, such as acknowledgments from their hairdressers, to improve their experience.

[1215] The present invention is an advanced system that combines a system that accurately conveys the ideal hairstyle and color when visiting a beauty salon with an emotion engine that recognizes the user's emotions. Specific embodiments executed by the user, terminal, and server are described below.

[1216] User Actions

[1217] Users install and launch a dedicated application on their smartphone. This application is available for iOS and Android platforms. Users can take a photo of their face using the application's camera function or select a photo they have already taken from their gallery. They can then upload the selected photo within the app. This upload causes the device to send the photo to the server.

[1218] Terminal handling

[1219] The device converts the selected photo into a data packet and sends it over the Internet to a server, which temporarily stores the image in an Amazon Web Services (AWS) S3 bucket. The user also inputs desired hairstyle and color information using an in-app chat interface (e.g., Dialogflow).

[1220] Server Processing

[1221] The server analyzes the received photo and the user's instructions. The natural language processing engine (e.g., NLTK) used analyzes the user's instructions and provides them as prompts to a generative AI model (e.g., StyleGAN2). The generative AI model generates an image based on the user's face photo, with the hairstyle and color modified. This image is temporarily stored in a database (e.g., AWS RDS).

[1222] An example of a prompt sentence would be "Shoulder-length, swept-back bangs, ash gray" and sent to the generation AI.

[1223] Image generation and review

[1224] The image processed by the generative AI model is sent back to the device from the server and displayed in a format that the user can check within "BeautyApp." The user can check the processed image and input instructions for fine-tuning as needed, such as "Make the bangs a little shorter."

[1225] The role of the emotional engine

[1226] While the user is viewing the image, an emotion engine (e.g., Microsoft Azure's Emotion API) analyzes the user's facial expressions and voice. If the user shows a dissatisfied expression, the emotion engine automatically generates and suggests alternatives. These suggestions are presented to the user through the chat interface.

[1227] Final confirmation and submission

[1228] When the user taps the "OK" button on the image they are finally satisfied with, the device sends the final edited image to the server. The server then sends the final edited image and the results of the user's emotional analysis to the salon. Based on this information, the hairdresser can understand the user's wishes and mood before the treatment. The device then receives a receipt confirmation and notifies the user.

[1229] As a concrete example, a user selects and uploads a selfie and inputs instructions such as "I want shoulder-length hair, swept-back bangs, and an ash gray color." The server analyzes these instructions and generates an edited image using a generative AI model. The user then reviews the image, and an emotion engine analyzes the user's facial expressions and suggests alternatives if the user is dissatisfied. This process is repeated until the final image the user is satisfied with is sent to the hairdresser, and a confirmation of receipt is obtained.

[1230] The present invention allows users to accurately communicate their ideal hairstyle and color and achieve the desired results.

[1231] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1232] Step 1:

[1233] The user installs and launches a dedicated application on their smartphone. The application prompts the user to register and configure the settings. The user then takes a photo of their face or selects an existing photo. The selected or taken photo is the input data.

[1234] Step 2:

[1235] The device converts the selected photo into a data packet and sends it to the server over the Internet. This process includes the following specific operations:

[1236] Loading images

[1237] Image data compression and encryption

[1238] Sending to the server

[1239] The transmitted image data is the output.

[1240] Step 3:

[1241] The server temporarily stores the received image data in an AWS S3 bucket. The stored image data is the input. The server confirms receipt of the image data and returns an output indicating that the data has been saved.

[1242] Step 4:

[1243] Users use the app's chat interface to input their desired hairstyle and color, such as "shoulder-length, swept-back bangs, ash gray." This is the input data.

[1244] Step 5:

[1245] The terminal assembles the user's instructions into a data packet and sends it to the server. The instructions are input. The server then sends the received instructions to the analysis engine and outputs the analysis results.

[1246] Step 6:

[1247] The server analyzes the received instructions using a natural language processing engine (NLTK). The analysis result is the input. The analyzed instructions (prompt sentence) are the output.

[1248] Step 7:

[1249] The server supplies a prompt to a generative AI model (StyleGAN2) to generate an image. The input includes the prompt and a saved facial photo. The generative AI model generates an image with the hairstyle and color applied based on the user's instructions. The generated image is the output.

[1250] Step 8:

[1251] The server temporarily stores the generated image in the AWS database and returns it to the device. Amazon S3 and CloudFront are used for the return. The generated image is the input data. A return completion message is the output.

[1252] Step 9:

[1253] The device displays the generated image to the user. The user checks the displayed image and inputs instructions for fine-tuning as needed. For example, specific instructions such as "Make the bangs a little shorter" are input data. The displayed image is the output.

[1254] Step 10:

[1255] The device sends additional instructions from the user to the server again. The sent instructions are input data. The server then supplies a new prompt to the generation AI in the same way and outputs the recreated image.

[1256] Step 11:

[1257] While the user is viewing the image, the emotion engine (Microsoft Azure's Emotion API) analyzes the user's facial expressions and voice. The analyzed facial expressions and voice are the input data. If the user has a dissatisfied expression, the emotion engine automatically generates and proposes an alternative. This proposal is the output.

[1258] Step 12:

[1259] When the user taps the "OK" button on the image they are finally satisfied with, the device sends the final processed image to the server. The sent image is the input data.

[1260] Step 13:

[1261] The server sends the final processed image and the user's sentiment analysis results to the hair salon. The input data is the hairdresser's contact information. The output is a confirmation message after the transmission is complete.

[1262] Step 14:

[1263] The hairdresser performs the treatment based on the received processed image and the emotion analysis results. Finally, a receipt confirmation from the hairdresser is sent to the user, and the user is notified of the information. The notification data is the output.

[1264] (Application example 2)

[1265] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1266] With conventional virtual try-on systems, it is difficult to accurately simulate the hairstyle and color desired by the user. The lack of functionality to analyze and suggest fine-tuning requests and emotions made it difficult for users to achieve a final result that satisfied them. Furthermore, the inability to send the final edited image to an expert and provide real-time feedback to the user made it difficult to provide optimal service.

[1267] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1268] In this invention, the server includes means for receiving an image taken or selected by the user, means for inputting instructions for the user's desired hairstyle and color, means for processing the image based on the received image in accordance with the user's instructions using generative artificial intelligence, means for displaying the processed image to the user and analyzing the user's facial expressions and voice to recognize emotions, means for receiving the user's instructions for fine-tuning and processing again, and means for saving the final processed image and sending it to an expert. This allows the server to simulate hairstyles and colors while taking the user's emotions into consideration and share the results with the expert in real time, thereby enabling the user to ultimately achieve results that they are satisfied with.

[1269] "Means for receiving images taken or selected by the user" refers to the function that allows the user to upload images they have taken themselves or images they already have to the device, and for the system to receive them.

[1270] "Means for users to input instructions for their desired hairstyle and color" refers to a function that allows users to input their desired hairstyle and color via text or voice and transmit that information to the system.

[1271] "Means for processing an image based on the received image in accordance with the user's instructions using generative artificial intelligence" refers to a function that uses artificial intelligence technology to process the image based on the received image data and the user's instructions.

[1272] "Means of recognizing emotions by displaying a processed image to the user and analyzing the user's facial expressions and voice" refers to a function that displays a processed image to the user, analyzes the user's facial expressions and voice to understand their emotions, and reflects them in the system.

[1273] The "means for receiving a fine adjustment instruction from the user and processing the image again" is a function for receiving a fine adjustment request from the user and processing the image again based on the request.

[1274] The "means for saving the final processed image and sending it to an expert" is a function for saving the final processed image that the user is satisfied with and transferring it to an expert.

[1275] The present invention provides a system that allows a user to simulate hairstyles and colors in an autonomous vehicle and transmit the results to an expert. This system is realized using the following means.

[1276] User Actions

[1277] Using the interface inside the autonomous vehicle, users can take a photo of themselves or select a photo they have already taken from their gallery, then upload the selected photo to a dedicated application, which then sends the photo to a server.

[1278] Server Processing

[1279] The server receives the photo sent from the device and temporarily stores it. It provides an interface for the user to input instructions for the desired hairstyle and color. The user's instructions are sent to the server via the device, and the server analyzes them and provides them to the generative AI model. The generative AI model generates an image with the hairstyle and color applied based on the user's specifications.

[1280] Image generation and display

[1281] The image processed by the generative AI model is sent from the server to the device and displayed for the user to review. As the user reviews the image, the emotion engine analyzes their facial expressions and voice to recognize their emotions. If the user is not satisfied, it will suggest an alternative. Furthermore, it can accept instructions for fine-tuning and process the image again. This process is repeated until the user is finally satisfied.

[1282] Final confirmation and submission

[1283] When the user finally taps the "Decide" button on the image they are satisfied with, the device prepares to send the final edited image to the expert. The server forwards the edited image to the expert's designated contact and obtains the expert's confirmation of receipt. The results of the user's sentiment analysis are also sent to the expert, allowing the expert to understand the user's mood and preferences before providing the service.

[1284] Hardware and software used

[1285] Hardware: Camera, large display, smartphone, head-mounted display

[1286] Software: OpenCV (face detection and image processing), dlib (face detection), Keras (emotion recognition model), generative AI model

[1287] As a concrete example, a user can input instructions through an in-car interface based on a selfie they have uploaded, such as "I want shoulder-length hair, swept-back bangs, and an ash gray color." The server analyzes these instructions and processes the image using a generative AI model. When the user reviews the image, the emotion engine analyzes the user's facial expressions and automatically suggests "making the bangs a little shorter." The user then agrees to the reprocessing, and the server processes the image again using the generative AI model. This process is repeated, and the final image that the user is satisfied with is sent to the expert. The expert then confirms receipt and provides the service, taking into account the results of the emotion analysis.

[1288] Prompt Sentence Examples

[1289] "Long hair, blonde."

[1290] "Shoulder-length hair, swept-back bangs, ash gray."

[1291] As described above, the present invention enables simulation of hairstyles and colors that take into account the user's emotions within an autonomous vehicle, making it possible to smoothly communicate information to experts.

[1292] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1293] Step 1:

[1294] The user launches a dedicated application in the self-driving vehicle and takes a photo of their face or selects a photo they have already saved.

[1295] Input: A photo of your face taken or selected by the user

[1296] Output: Photo data

[1297] How it works: Take a photo using your smartphone or car camera, or select a photo from your gallery and upload it to the dedicated application.

[1298] Step 2:

[1299] The terminal transmits the uploaded photo data to the server.

[1300] Input: Photo data

[1301] Output: Photo data sent to the server

[1302] How it works: Your device uploads photo data to a server via your internet connection.

[1303] Step 3:

[1304] The server temporarily stores the received photo data and provides an interface for the user to input instructions for the desired hairstyle and color.

[1305] Input: Photo data

[1306] Output: Instruction input interface

[1307] How it works: The server stores the photo data in a database and prompts the user for input for the next processing step.

[1308] Step 4:

[1309] The user inputs the desired hairstyle and color and sends the instructions to the server.

[1310] Input: Hair style and color instructions (prompt text)

[1311] Output: User instructions

[1312] How it works: The user enters their desired hairstyle and color via text or voice into a field within the application and submits that information.

[1313] Step 5:

[1314] The server analyzes the user's instructions and supplies them to the generative AI model, which then generates an image with the hairstyle and color applied based on the received photo data and the user's instructions.

[1315] Input: Photo data, user instructions

[1316] Output: Processed image

[1317] How it works: The server analyzes the text of the user's instructions and passes them to a generative AI model to process the image.

[1318] Step 6:

[1319] The server transmits the generated processed image to the terminal and displays it to the user.

[1320] Input: processed image

[1321] Output: Image sent to user device

[1322] How it works: The server sends the generated image data to the user's device, which then displays the image.

[1323] Step 7:

[1324] The user can review the edited image and input instructions for fine-tuning, while the emotion engine analyzes the user's facial expressions and voice to recognize their emotions.

[1325] Input: User emotion data, fine-tuning instructions

[1326] Output: User emotion data and fine-tuning content

[1327] How it works: The application captures the user's facial expressions and voice using a camera and microphone, which the emotion engine analyzes. The user then inputs fine-tuning instructions.

[1328] Step 8:

[1329] The server receives the user's fine-tuning instructions and emotion data and requests the generative AI model to process it again.

[1330] Input: Fine-tuning instructions, user emotion data

[1331] Output: Reprocessed image

[1332] How it works: The server analyzes the emotion data and fine-tuning instructions, then passes new instructions to the generative AI model for reprocessing.

[1333] Step 9:

[1334] When the user finally taps the "Decide" button on the image they are satisfied with, the device prepares the final edited image for sending to the expert and notifies the server.

[1335] Input: Final processed image

[1336] Output: Prepared for sending to a specialist

[1337] How it works: The user taps the "OK" button, and the device notifies the server of the final processed image.

[1338] Step 10:

[1339] The server forwards the final processed image to the expert's designated contact and obtains the expert's acknowledgement of receipt. It also sends the user's emotion data to the expert.

[1340] Input: Final processed image, user emotion data

[1341] Output: Send to expert, acknowledge receipt

[1342] How it works: The server sends the final processed image and emotion data to the expert, receives confirmation data, and provides feedback to the user.

[1343] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1344] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1345] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[1346] [Fourth embodiment]

[1347] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[1348] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1349] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1350] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[1351] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1352] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1353] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1354] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[1355] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1356] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1357] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1358] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1359] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1360] The present invention is a system that allows users to accurately communicate their ideal hairstyle and color when visiting a beauty salon. This system operates in cooperation with the user, terminal, and server elements. A specific example is shown below.

[1361] User Actions

[1362] The first step in using this system is for the user to install and launch the dedicated application. Next, the user takes a photo of their face or selects a photo they have already taken from their gallery. The selected photo is uploaded within the app. This upload action causes the device to send the user's photo to the server.

[1363] Server Processing

[1364] The server receives the photos sent from the device and temporarily stores them. The user uses a chat interface to input instructions for the desired hairstyle and color. The user's instructions are sent to the server via the device, and the server analyzes them. The analyzed instructions are then provided to a generation AI, which then generates an image with the hairstyle and color modified based on the user's specifications.

[1365] Image generation and display

[1366] The image processed by the generation AI is sent from the server to the device and displayed for the user to review. The user reviews the processed image and inputs fine-tuning instructions as needed. These fine-tuning instructions are sent back to the server, which then requests further processing from the generation AI. Fine-tuning is carried out and a new image is generated. This process is repeated until the user is satisfied.

[1367] Final confirmation and submission

[1368] If the user is satisfied with the final image, they tap the "OK" button. This causes the device to prepare to send the final processed image to the server, which then forwards the image to the designated hairdresser. The hairdresser then confirms receipt, and this confirmation information is sent back to the user via the server.

[1369] Specific examples

[1370] As a concrete example, consider a scenario in which a user launches a dedicated app and selects and uploads a selfie. After this photo is sent to the server, the user uses a chat interface to input instructions such as, "Shoulder-length hair, swept-back bangs, and ash gray color." The server analyzes these instructions and sends a processing request to the generation AI. The generation AI generates an image based on the user's face photo with the hairstyle and color applied, and sends this back to the user. After the user confirms this, they can further input fine-tuning instructions such as, "Make the bangs a little shorter." The server processes the image again using the generation AI, and finally sends the final image they are satisfied with to the hairdresser. As a result, the user can accurately communicate their ideal hairstyle and achieve the desired results.

[1371] The processing flow will be explained below.

[1372] Program processing steps

[1373] 1. User photo submission

[1374] Step 1: Select and upload a photo

[1375] User

[1376] Open the application.

[1377] Tap the "Select Photo" button to choose a photo of your face from your device's gallery or take a new one.

[1378] Tap the "Upload" button to send the photo you selected or took.

[1379] Step 2: Prepare your photos for sending

[1380] Terminal

[1381] Temporarily store photos uploaded by users.

[1382] Sends a request to the server to prepare the photo for sending.

[1383] Step 3: Receive and save photos

[1384] server

[1385] Receives photo data sent from the device.

[1386] Save the photos in a database.

[1387] A confirmation to save the photo will be sent to your device.

[1388] ---

[1389] 2. Specify hairstyle and color

[1390] Step 4: Enter instructions

[1391] User

[1392] Use the app's chat interface to input your desired cut (e.g., "shoulder-length") and color (e.g., "ash gray").

[1393] Tap the "Send" button to send the instructions.

[1394] Step 5: Sending instructions

[1395] Terminal

[1396] Receive and temporarily store instructions from the user.

[1397] The instruction is sent to the server.

[1398] Step 6: Parse instructions and input them into the generative AI

[1399] server

[1400] Analyze the instructions received from the terminal.

[1401] The saved photos and instructions are input into the generative AI system.

[1402] The generative AI generates images with edited hairstyles and colors based on the user's instructions.

[1403] ---

[1404] 3. Instructions for fine-tuning and their reflection

[1405] Step 7: Check the processed image and give instructions for fine adjustments

[1406] User

[1407] The processed image returned from the server is checked within the application.

[1408] If you need to make any tweaks, enter new instructions (e.g., "Make the bangs a little shorter") and tap the "Send" button.

[1409] Step 8: Sending fine-tuning instructions

[1410] Terminal

[1411] Receives new instructions and sends them to the server.

[1412] Step 9: Applying the tweaks

[1413] server

[1414] Re-input the received fine-tuning instructions into the generation AI.

[1415] The generative AI readjusts the image and generates a new one.

[1416] The adjusted image is sent to the device.

[1417] This process is repeated until the user is satisfied.

[1418] ---

[1419] 4. Sending the final image

[1420] Step 10: Check the final image

[1421] User

[1422] Finally, check the image of the hairstyle you are satisfied with in the application.

[1423] Tap the "Decide Now" button.

[1424] Step 11: Prepare your images for delivery

[1425] Terminal

[1426] Prepare the final processed image for sending to the beauty salon.

[1427] Step 12: Send to salon

[1428] server

[1429] Send a message to your salon contact with the final processed image.

[1430] The hairdresser's confirmation of receipt is obtained and the status is fed back to the terminal.

[1431] Save the transmission history in the database.

[1432] Example 1

[1433] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1434] When visiting a hair salon, it is difficult for users to accurately communicate their ideal hairstyle and color. In particular, a communication gap between the user and the hairdresser makes it difficult for users to achieve a satisfactory result. For this reason, a system is needed that allows users to visually confirm their ideal style and accurately communicate it to the hairdresser.

[1435] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1436] In this invention, the server includes a means for receiving an image taken or selected by the user, a means for inputting instructions for the user's desired hairstyle and color, a means for processing the image based on the received image in accordance with the user's instructions using a generative artificial intelligence, a means for displaying the processed image to the user, receiving fine adjustment instructions from the user and processing it again, and a means for saving the final processed image and sending it to the hairdresser. This allows the user to visually confirm their ideal hairstyle and color and accurately communicate them to the hairdresser.

[1437] A "user" is an individual who uses the system to specify their ideal hairstyle and color and actually generate an image.

[1438] "Photo or Selected Image" means an image of a user's face taken or selected from existing images and uploaded to the system.

[1439] "Instructions" refers to the specific input the user makes about the hairstyle and color they desire.

[1440] "Generative AI" refers to an AI technology for processing images based on user instructions, such as generative models.

[1441] "Means for processing images" refers to the functionality for using the generated AI model to edit or modify images taken or selected based on user instructions.

[1442] "Fine-tuning instructions" refer to requests for additional corrections or changes made by the user to the displayed edited image.

[1443] The "final edited image" refers to the final image that the user is satisfied with after making fine adjustments.

[1444] A "hairdresser" is a professional who receives the final edited image sent by the user and performs the actual hairstyle and coloring based on it.

[1445] "Server" refers to the core computer system that receives and stores user images, analyzes instructions, and works with the generating AI.

[1446] The present invention is a system that allows users to accurately communicate their ideal hairstyle and color when visiting a beauty salon. This system achieves its purpose by having the user, terminal, and server work together.

[1447] User Actions

[1448] Users install and launch the app from the App Store or Google Play. After creating an account and logging in, they can either take a selfie using the app's photo feature or select an existing photo from their gallery. After selecting, the device uploads the photo to the server. For example, if a user takes a selfie and taps the upload button in the app, the device compresses the photo and sends it to the server using a secure protocol.

[1449] Server Processing

[1450] The server receives the image sent by the user and temporarily stores it. The user then inputs their desired hairstyle and color through the chat interface. For example, they might input specific instructions such as "shoulder-length hair, swept-back bangs, and ash gray color." The instructions are sent via the user's device to the server, which analyzes them. The analyzed information is then provided to the generation AI, which then processes the image based on the user's instructions.

[1451] Examples of generative AI that can be used include Stable Diffusion and DALL-E. The server sends appropriate prompts to the generative AI model to generate images. For example, the following prompts are used:

[1452] "Generate an image of the user's face with shoulder-length hair, swept-back bangs, and ash gray hair."

[1453] Image generation and display

[1454] The image processed by the generation AI is sent from the server to the device and displayed for the user to review. At this point, the user can review the generated image and input fine-tuning instructions as needed. For example, they can input fine-tuning instructions such as "Make the bangs a little shorter." The instructions sent back to the server are analyzed, and a new image is generated by sending additional prompts to the generation AI. The image is then sent to the user again, and this process is repeated until the user is satisfied.

[1455] Final confirmation and submission

[1456] When the user finally hits the "OK" button on the image they are satisfied with, the device prepares to send the final processed image to the server, which then forwards the image to a dedicated hairdresser. After the hairdresser confirms receipt, the confirmation information is sent back to the user via the server.

[1457] This system allows users to visually confirm their ideal hairstyle and color and accurately communicate it to the hairdresser, which is expected to result in the user achieving the desired results.

[1458] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1459] Step 1: Initial Setup

[1460] The user installs and launches the dedicated application, creates an account, and logs in.

[1461] Input: Installed apps, user information (email address, password)

[1462] Output: The app launches and the user is logged in.

[1463] Step 2: Upload your photo

[1464] The user takes a photo of their face using the app's photo function or selects an existing photo, and the device uploads the selected photo to the server.

[1465] Input: Captured or selected image

[1466] Output: The image is uploaded to the server.

[1467] Specific operation: For example, when you tap the camera button in the app, the camera starts up, you take a picture, check it, and then tap the upload button. The device compresses the image and sends it to the server using the HTTPS protocol.

[1468] Step 3: User prompts

[1469] Users input their desired hairstyle and color through a chat interface, and the device sends the user's instructions to the server.

[1470] Input: User's hairstyle and color instructions (e.g., "Shoulder-length, swept-back bangs, ash gray color")

[1471] Output: The instructions are sent to the server.

[1472] Specific operation: The user enters text in the chat interface and taps the send button. The device sends the text data to the server.

[1473] Step 4: Instruction analysis and image generation

[1474] The server receives and analyzes the user's instructions. The server sends the analysis results to the generation AI, which issues instructions as prompts. The generation AI generates an image based on the prompts.

[1475] Input: User's instructions (text), user's image data

[1476] Output: processed image

[1477] Specific operation: The server uses a natural language processing engine to analyze the instructions, generate specific prompts such as "Hair length: shoulder length," "Bangs: swept-back," and "Color: ash gray," and sends them to the generation AI. The generation AI then generates an image based on this and sends it back to the server.

[1478] Step 5: Image generation and display

[1479] The image generated by the AI ​​is received and stored by the server, which then sends the image to the device and displays it for the user to view.

[1480] Input: Generated image

[1481] Output: The image is displayed on the user's device.

[1482] Specific operation: The server temporarily saves the generated image file in a database, generates a URL and sends it to the device, where the image is displayed and the user is asked to confirm it.

[1483] Step 6: Fine-tuning instructions

[1484] The user checks the generated image and inputs fine-tuning instructions as needed. The device then sends the fine-tuning instructions to the server.

[1485] Input: User fine-tuning instructions (e.g., "Make my bangs a little shorter")

[1486] Output: Fine-tuning instructions are sent to the server.

[1487] Specific operation: The user enters new instructions in the chat interface and taps the send button. The device sends the text data to the server again.

[1488] Step 7: Reprocessing and display

[1489] The server then sends a new prompt to the AI ​​generator, requesting it to generate a finely tuned image. The AI ​​generator generates a new image, which the server stores and sends to the device.

[1490] Input: New fine-tuning instructions, original image data

[1491] Output: New processed image

[1492] What it does: The server analyzes the new instructions and sends prompts to the generation AI to generate new image data, which is then sent back to the device and displayed to the user.

[1493] Step 8: Final Review and Submission

[1494] If the user is satisfied with the final image, they tap the "OK" button. The device prepares and sends the final image to the server. The server then forwards the final image to the designated hairdresser. The hairdresser confirms receipt, and the server returns the confirmation information to the user.

[1495] Input: Final processed image

[1496] Output: Final image is sent to the hairdresser, confirmation information is fed back to the user.

[1497] Specific operation: When the user taps the "OK" button, the device compresses the final image data and sends it to the server. The server then forwards the image to the hairdresser, receives confirmation from the hairdresser, and sends feedback to the device.

[1498] (Application example 1)

[1499] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1500] There is a problem that it is difficult for hair salon customers to accurately communicate their ideal hairstyle and color to the hairdresser. This problem manifests itself in the form of miscommunication and dissatisfaction with the finished product due to differences in image. In addition, insufficient reservation management and customer service upon arrival are factors that detract from the customer experience at hair salons. To solve these issues, a system is needed that allows users to virtually try on their ideal hairstyle from home or on the go and share their specific image with the hairdresser.

[1501] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1502] In this invention, the server includes a means for receiving images taken or selected by the user, a means for inputting instructions for the user's desired hairstyle and color, a means for processing the received image according to the user's instructions using a generative AI model, a means for displaying the processed image to the user and receiving fine-tuning instructions from the user for further processing, a means for saving the final processed image and sending it to the hairdresser, and a means for managing the user's appointments. This allows users to try out their ideal hairstyle from home or on the go and share their specific image with the hairdresser, facilitating communication and improving satisfaction with the finished product. It also streamlines appointment management and customer service during visits, improving the overall customer experience.

[1503] "User" refers to an individual who uses the system to try out their ideal hairstyle and color and input their instructions.

[1504] "Image receiving means" refers to a device or system that has the function of uploading and receiving photos taken or selected by a user to a server through an application.

[1505] "Instruction input means" refers to a device or system having an interface for a user to input details of the hairstyle or color desired by the user via text or voice.

[1506] "Generative AI model" refers to an algorithm or system that uses artificial intelligence technology to generate or modify images based on information provided by users.

[1507] "Image processing means" refers to a device or system that edits and processes images in accordance with user instructions based on the received image using a generative AI model.

[1508] The term "fine-tuning instruction receiving means" refers to a device or system that has the function of displaying the processed image, receiving additional instructions from the user, and reprocessing the image based on those instructions.

[1509] "Final image storage means" refers to a device or system that stores the final processed image that the user is satisfied with.

[1510] The term "means for transmitting to a hairdresser" refers to a device or system that has the function of transmitting the final processed image to a designated hairdresser.

[1511] "Reservation management means" refers to a device or system that has the function of managing user reservation information and adjusting reservations and schedules.

[1512] "Reprocessing interface" refers to an interface that allows a user to review the generated image and input instructions for reprocessing, if necessary.

[1513] A "prompt" is a text that contains specific instructions for a generative AI model, and refers to an instruction used to improve the accuracy of image generation and processing.

[1514] "Professionals" refers to hairdressers and hairstylists who receive the images and instructions sent by the system and actually perform the hairstyle and coloring.

[1515] This invention is a system that can accurately convey the ideal hairstyle and color when visiting a beauty salon. This system works in cooperation with the user's device, a server, and a generative AI model.

[1516] User Actions

[1517] Users first install the app on their smartphone and launch it. Then, they take a photo of themselves or select a photo they have already taken from their gallery. The selected photo is then uploaded within the app and sent to the server.

[1518] Server Processing

[1519] The server receives the user's photo sent from the device and temporarily stores it. When the user inputs instructions for the desired hairstyle or color using the chat interface, the instructions are sent to the server via the device. The server analyzes the instructions and provides them as prompts to the generative AI model.

[1520] Image generation using generative AI models

[1521] The generative AI model processes the received image based on the user's instructions. For example, if the instruction is "short bob, black hair, swept bangs," the generative AI model will generate an image based on this information.

[1522] Example prompt sentence:

[1523] "Generate an image of a user with a short bob, dark hair, and swept-back bangs."

[1524] Image confirmation and fine adjustment

[1525] The processed image is sent from the server to the device and displayed for the user to review. The user can then confirm that the image is as desired and input fine-tuning instructions as needed. These fine-tuning instructions are sent back to the server, which requests further processing from the generative AI model. This process is repeated until the user is satisfied.

[1526] Final confirmation and submission

[1527] If the user is satisfied with the final image, they tap the "OK" button. The device prepares to send the final processed image to the server, which then forwards the image to the designated hairdresser. The hairdresser confirms receipt, and the confirmation information is sent back to the user via the server.

[1528] Reservation management and store visit support

[1529] In addition, the system is equipped with a function to manage user reservation information, which allows for smooth adjustment of user reservations and streamlines customer service when they visit the store.

[1530] Specific examples

[1531] For example, a user launches a dedicated app and selects and uploads a selfie. After the photo is sent to the server, the user uses a chat interface to input instructions such as, "I want shoulder-length hair, swept-back bangs, and an ash gray color." The server analyzes these instructions and sends a request to the generative AI model to edit the image. The generative AI model generates an image based on the user's face photo with the hairstyle and color applied, and sends this back to the user. After the user confirms the image, the user can again input instructions for fine-tuning, such as, "Make the bangs a little shorter." The server then requests further editing from the generative AI model and generates a new image. This process is repeated until the user is satisfied. The final image is sent to the hairdresser, and appointment information is managed, allowing the user to get the hairstyle they want.

[1532] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1533] Step 1:

[1534] The user installs and launches the dedicated app on their smartphone. Next, they take a photo of their face or select a previously taken photo from the gallery. The selected photo is uploaded within the app. The user's photo is provided as input, and the photo data is sent to the application server. The server then receives the user's image data.

[1535] Step 2:

[1536] The server receives the user's photos sent from the device and temporarily stores them. Once the photo data is received, it prepares to analyze it.

[1537] Step 3:

[1538] The user uses a chat interface to input instructions for the desired hairstyle and color. The instructions are provided in text format as input, and the instructions are sent from the device to a server, which parses the instructions and generates a prompt for the generative AI model.

[1539] Step 4:

[1540] The server sends the generated prompt to the generative AI model. Specifically, the prompt is used to make the generative AI model generate an image with the hairstyle and color applied. The prompt and photo data are provided as input, and the generative AI model outputs an edited image based on these.

[1541] Step 5:

[1542] The processed image generated by the generative AI model is sent from the server to the device, where the server receives the generated image and returns it in a format that can be displayed to the user.

[1543] Step 6:

[1544] The user checks the generated image, and if it is not as desired, inputs instructions for fine-tuning in the chat interface. The fine-tuning instructions are provided as input from the user, and the instruction data is sent from the terminal to the server.

[1545] Step 7:

[1546] The server receives the user's fine-tuning instructions and sends them to the generative AI model as a new prompt. The fine-tuning instructions are provided as input, and the generative AI model outputs a newly processed image.

[1547] Step 8:

[1548] The regenerated processed image is sent from the server to the user's device, where the user can review it again. This process is repeated until the user is satisfied.

[1549] Step 9:

[1550] If the user is satisfied with the final image, he / she presses the "OK" button, providing the final processed image as input, which is then sent to the server.

[1551] Step 10:

[1552] The server stores the final processed image and sends it to the hairdresser. The final processed image is provided as input and forwarded to the hairdresser. The hairdresser confirms receipt, and the confirmation information is fed back to the user through the server.

[1553] Step 11:

[1554] The system manages the user's reservation information and provides hairdressers with information to ensure a smooth customer experience when they visit. The user's reservation information is provided as input and shared with the hairdresser.

[1555] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1556] The present invention is an advanced system that combines a system that accurately conveys the ideal hairstyle and color when visiting a beauty salon with an emotion engine that recognizes the user's emotions. Specific embodiments executed by the user, terminal, and server are described below.

[1557] User Actions

[1558] Users install and launch the app, then take a photo of themselves or select a photo they have already taken from their gallery, and upload the photo within the app. This upload process causes the device to send the photo to the server.

[1559] Server Processing

[1560] The server receives the photos sent from the device and temporarily stores them. Users use a chat interface to input instructions for their desired hairstyle and color. The user's instructions are sent via the device to the server, which analyzes them. The analyzed instructions are then provided to a generation AI, which then generates an image with the hairstyle and color modified based on the user's specifications.

[1561] Image generation and display

[1562] The image processed by the generation AI is sent from the server to the device and displayed for the user to review. The user reviews the processed image and enters instructions for fine-tuning as needed. These instructions are sent back to the server, which then requests further processing from the generation AI. Fine-tuning is carried out and a new image is generated. This process is repeated until the user is satisfied.

[1563] The role of the emotional engine

[1564] As the user reviews the image, the emotion engine analyzes the user's facial expressions and voice to recognize their emotions. For example, if the user looks dissatisfied, the emotion engine automatically suggests an alternative. The emotion analysis results are sent to the server and reflected in the next step of processing.

[1565] Final confirmation and submission

[1566] When the user finally taps the "OK" button on the image they are satisfied with, the device prepares to send the final edited image to the salon. The server forwards this edited image to the salon's designated contact and obtains the hairdresser's confirmation of receipt. The results of the user's sentiment analysis are also sent to the hairdresser, allowing the hairdresser to understand the user's mood and preferences before the treatment. This information is then fed back to the user.

[1567] Specific examples

[1568] As a concrete example, we will show a scene in which a user launches a dedicated application and selects and uploads a selfie. After the photo is sent to the server, the user uses a chat interface to input instructions such as, "I want shoulder-length hair, swept-back bangs, and an ash gray color." The server analyzes these instructions and sends a processing request to the generation AI. The generation AI generates an image based on the user's face photo, applying the hairstyle and color, and sends this back to the user. When the user reviews the image, the emotion engine analyzes the user's facial expression. If the user is not satisfied, it automatically suggests, "Make the bangs a little shorter." The user accepts, and the server uses the generation AI to process the image again. This process is repeated, and finally, the image that the user is satisfied with is sent to the hairdresser. The hairdresser confirms receipt and performs the treatment taking the emotion analysis results into account. As a result, the user can accurately communicate their ideal hairstyle and achieve the desired results.

[1569] The processing flow will be explained below.

[1570] Program processing steps

[1571] 1. User photo submission

[1572] Step 1: Select and upload a photo

[1573] User

[1574] Open the application.

[1575] Tap the "Select Photo" button to choose a photo of your face from your device's gallery or take a new one.

[1576] Tap the "Upload" button to send the photo you selected or took.

[1577] Step 2: Prepare your photos for sending

[1578] Terminal

[1579] Temporarily store photos uploaded by users.

[1580] Sends a request to the server to prepare the photo for sending.

[1581] Step 3: Receive and save photos

[1582] server

[1583] Receives photo data sent from the device.

[1584] Save the photos in a database.

[1585] A confirmation to save the photo will be sent to your device.

[1586] ---

[1587] 2. Specify hairstyle and color

[1588] Step 4: Enter instructions

[1589] User

[1590] Use the app's chat interface to input your desired cut (e.g., "shoulder-length") and color (e.g., "ash gray").

[1591] Tap the "Send" button to send the instructions.

[1592] Step 5: Sending instructions

[1593] Terminal

[1594] Receive and temporarily store instructions from the user.

[1595] The instruction is sent to the server.

[1596] Step 6: Parse instructions and input them into the generative AI

[1597] server

[1598] Analyze the instructions received from the terminal.

[1599] The saved photos and instructions are input into the generative AI system.

[1600] The generative AI generates images with edited hairstyles and colors based on the user's instructions.

[1601] ---

[1602] 3. Instructions for fine-tuning and their reflection

[1603] Step 7: Checking edited images and analyzing sentiment

[1604] User

[1605] The processed image returned from the server is checked within the application.

[1606] During this process, the emotion engine analyzes the user's facial expressions and voice to recognize their emotions.

[1607] Step 8: Emotional Feedback

[1608] server

[1609] The emotion engine receives the user's recognized emotions and presents the next suggestion to the user based on the analysis results.

[1610] For example, if the person looks dissatisfied, the system will make a suggestion such as, "Would you like your bangs cut a little shorter?"

[1611] Step 9: Enter and submit fine-tuning instructions

[1612] User

[1613] Accept the suggestions based on the sentiment analysis results, or enter new fine-tuning instructions (e.g., "Make it a little brighter").

[1614] Send the instructions by tapping the "Send" button.

[1615] Step 10: Analyze and refine the fine-tuning instructions

[1616] server

[1617] Re-input the received fine-tuning instructions into the generation AI.

[1618] The generative AI readjusts the image and generates a new one.

[1619] The adjusted image is sent to the device.

[1620] This process is repeated until the user is satisfied.

[1621] ---

[1622] 4. Sending the final image

[1623] Step 11: Check the final image

[1624] User

[1625] Finally, check the image of the hairstyle you are satisfied with in the application.

[1626] Tap the "Decide Now" button.

[1627] Step 12: Prepare your images for delivery

[1628] Terminal

[1629] Prepare the final processed image for sending to the beauty salon.

[1630] Step 13: Send to salon

[1631] server

[1632] Send a message to your salon contact with the final processed image.

[1633] The hairdresser's confirmation of receipt is obtained and the status is fed back to the terminal.

[1634] Save the transmission history in the database.

[1635] Step 14: Sending sentiment analysis results to hairdressers

[1636] server

[1637] The results of the user's sentiment analysis are also sent to the hairdresser, allowing the hairdresser to understand the user's mood and preferences before the treatment.

[1638] Specific examples

[1639] As a concrete example, consider a scenario in which a user launches a dedicated application, selects and uploads a selfie. After the photo is sent to the server, the user enters instructions in the chat interface, such as "I want shoulder-length hair, swept-back bangs, and an ash gray color," and submits the image. The server analyzes the instructions and generates an edited image using generative AI. When the user reviews the image, the emotion engine suggests, "Maybe the bangs would be better if they were a little shorter." The user accepts the suggestion, and the server again fine-tunes the image using generative AI. The final image that the user is satisfied with is sent to the hairdresser, along with the results of the emotion analysis. This process allows the user to accurately communicate their ideal hairstyle and achieve the desired results.

[1640] Example 2

[1641] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1642] In conventional beauty salons, communication between customers and hairdressers has been hindered by the difficulty of accurately conveying the customer's desired hairstyle and color. Furthermore, if the customer is dissatisfied with the processed image, their feelings are not properly reflected, making it difficult to achieve the desired result. The present invention aims to solve these problems and provide a system that allows customers to accurately convey their desired hairstyle and color and achieve satisfactory results.

[1643] The specification process by the specification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes a means for receiving an image taken or selected by the user, a means for inputting instructions for the user's desired hairstyle and color, a means for processing the image based on the received image in accordance with the user's instructions using a generative artificial intelligence model, a means for displaying the processed image to the user and receiving fine-tuning instructions from the user for further processing, a means for saving the final processed image and sending it to the hairdresser, and a means for recognizing the user's emotions and automatically making suggestions based on the emotions. This allows the customer to accurately communicate their ideal hairstyle and color, ensure that their emotions are properly reflected in the generated image, and ultimately achieve a satisfactory result.

[1644] "User" refers to an individual who uses this system to give instructions about hairstyles and colors and check the results.

[1645] The "server" refers to the infrastructure responsible for processing user images and instructions, processing images using generative AI models, and recognizing user emotions.

[1646] A "generative AI model" is an artificial intelligence algorithm that processes hairstyles and colors based on a user's facial photo and instructions, specifically referring to technologies such as StyleGAN2.

[1647] An "emotion engine" refers to a software component that analyzes a user's facial expressions and voice and recognizes their emotions.

[1648] "Means for receiving images" refers to a function for transmitting an image taken or selected by a user to a server, and for the server to receive the image.

[1649] "Means for inputting instructions" refers to an interface through which a user can input information about the hairstyle and color they desire.

[1650] "Means of processing" refers to the function of processing received images based on user instructions using a generative AI model.

[1651] "Means for receiving fine-tuning instructions and reprocessing" refers to a function for receiving additional processing instructions from the user and reprocessing the image.

[1652] "Means for saving and sending to a hairdresser" refers to the function of saving the final processed image and sending the image to a hairdresser.

[1653] "Means for automatically making emotion-based suggestions" refers to the ability of the emotion engine to recognize the user's emotions and automatically make different suggestions based on the results.

[1654] "Final processed image" refers to the image generated in its final form, reflecting the user's wishes and fine-tuning instructions.

[1655] "Means for providing feedback" refers to features that notify users, such as acknowledgments from their hairdressers, to improve their experience.

[1656] The present invention is an advanced system that combines a system that accurately conveys the ideal hairstyle and color when visiting a beauty salon with an emotion engine that recognizes the user's emotions. Specific embodiments executed by the user, terminal, and server are described below.

[1657] User Actions

[1658] Users install and launch a dedicated application on their smartphone. This application is available for iOS and Android platforms. Users can take a photo of their face using the application's camera function or select a photo they have already taken from their gallery. They can then upload the selected photo within the app. This upload causes the device to send the photo to the server.

[1659] Terminal handling

[1660] The device converts the selected photo into a data packet and sends it over the Internet to a server, which temporarily stores the image in an Amazon Web Services (AWS) S3 bucket. The user also inputs desired hairstyle and color information using an in-app chat interface (e.g., Dialogflow).

[1661] Server Processing

[1662] The server analyzes the received photo and the user's instructions. The natural language processing engine (e.g., NLTK) used analyzes the user's instructions and provides them as prompts to a generative AI model (e.g., StyleGAN2). The generative AI model generates an image based on the user's face photo, with the hairstyle and color modified. This image is temporarily stored in a database (e.g., AWS RDS).

[1663] An example of a prompt sentence would be "Shoulder-length, swept-back bangs, ash gray" and sent to the generation AI.

[1664] Image generation and review

[1665] The image processed by the generative AI model is sent back to the device from the server and displayed in a format that the user can check within "BeautyApp." The user can check the processed image and input instructions for fine-tuning as needed, such as "Make the bangs a little shorter."

[1666] The role of the emotional engine

[1667] While the user is viewing the image, an emotion engine (e.g., Microsoft Azure's Emotion API) analyzes the user's facial expressions and voice. If the user shows a dissatisfied expression, the emotion engine automatically generates and suggests alternatives. These suggestions are presented to the user through the chat interface.

[1668] Final confirmation and submission

[1669] When the user taps the "OK" button on the image they are finally satisfied with, the device sends the final edited image to the server. The server then sends the final edited image and the results of the user's emotional analysis to the salon. Based on this information, the hairdresser can understand the user's wishes and mood before the treatment. The device then receives a receipt confirmation and notifies the user.

[1670] As a concrete example, a user selects and uploads a selfie and inputs instructions such as "I want shoulder-length hair, swept-back bangs, and an ash gray color." The server analyzes these instructions and generates an edited image using a generative AI model. The user then reviews the image, and an emotion engine analyzes the user's facial expressions and suggests alternatives if the user is dissatisfied. This process is repeated until the final image the user is satisfied with is sent to the hairdresser, and a confirmation of receipt is obtained.

[1671] The present invention allows users to accurately communicate their ideal hairstyle and color and achieve the desired results.

[1672] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1673] Step 1:

[1674] The user installs and launches a dedicated application on their smartphone. The application prompts the user to register and configure the settings. The user then takes a photo of their face or selects an existing photo. The selected or taken photo is the input data.

[1675] Step 2:

[1676] The device converts the selected photo into a data packet and sends it to the server over the Internet. This process includes the following specific operations:

[1677] Loading images

[1678] Image data compression and encryption

[1679] Sending to the server

[1680] The transmitted image data is the output.

[1681] Step 3:

[1682] The server temporarily stores the received image data in an AWS S3 bucket. The stored image data is the input. The server confirms receipt of the image data and returns an output indicating that the data has been saved.

[1683] Step 4:

[1684] Users use the app's chat interface to input their desired hairstyle and color, such as "shoulder-length, swept-back bangs, ash gray." This is the input data.

[1685] Step 5:

[1686] The terminal assembles the user's instructions into a data packet and sends it to the server. The instructions are input. The server then sends the received instructions to the analysis engine and outputs the analysis results.

[1687] Step 6:

[1688] The server analyzes the received instructions using a natural language processing engine (NLTK). The analysis result is the input. The analyzed instructions (prompt sentence) are the output.

[1689] Step 7:

[1690] The server supplies a prompt to a generative AI model (StyleGAN2) to generate an image. The input includes the prompt and a saved facial photo. The generative AI model generates an image with the hairstyle and color applied based on the user's instructions. The generated image is the output.

[1691] Step 8:

[1692] The server temporarily stores the generated image in the AWS database and returns it to the device. Amazon S3 and CloudFront are used for the return. The generated image is the input data. A return completion message is the output.

[1693] Step 9:

[1694] The device displays the generated image to the user. The user checks the displayed image and inputs instructions for fine-tuning as needed. For example, specific instructions such as "Make the bangs a little shorter" are input data. The displayed image is the output.

[1695] Step 10:

[1696] The device sends additional instructions from the user to the server again. The sent instructions are input data. The server then supplies a new prompt to the generation AI in the same way and outputs the recreated image.

[1697] Step 11:

[1698] While the user is viewing the image, the emotion engine (Microsoft Azure's Emotion API) analyzes the user's facial expressions and voice. The analyzed facial expressions and voice are the input data. If the user has a dissatisfied expression, the emotion engine automatically generates and proposes an alternative. This proposal is the output.

[1699] Step 12:

[1700] When the user taps the "OK" button on the image they are finally satisfied with, the device sends the final processed image to the server. The sent image is the input data.

[1701] Step 13:

[1702] The server sends the final processed image and the user's sentiment analysis results to the hair salon. The input data is the hairdresser's contact information. The output is a confirmation message after the transmission is complete.

[1703] Step 14:

[1704] The hairdresser performs the treatment based on the received processed image and the emotion analysis results. Finally, a receipt confirmation from the hairdresser is sent to the user, and the user is notified of the information. The notification data is the output.

[1705] (Application example 2)

[1706] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1707] With conventional virtual try-on systems, it is difficult to accurately simulate the hairstyle and color desired by the user. The lack of functionality to analyze and suggest fine-tuning requests and emotions made it difficult for users to achieve a final result that satisfied them. Furthermore, the inability to send the final edited image to an expert and provide real-time feedback to the user made it difficult to provide optimal service.

[1708] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1709] In this invention, the server includes means for receiving an image taken or selected by the user, means for inputting instructions for the user's desired hairstyle and color, means for processing the image based on the received image in accordance with the user's instructions using generative artificial intelligence, means for displaying the processed image to the user and analyzing the user's facial expressions and voice to recognize emotions, means for receiving the user's instructions for fine-tuning and processing again, and means for saving the final processed image and sending it to an expert. This allows the server to simulate hairstyles and colors while taking the user's emotions into consideration and share the results with the expert in real time, thereby enabling the user to ultimately achieve results that they are satisfied with.

[1710] "Means for receiving images taken or selected by the user" refers to the function that allows the user to upload images they have taken themselves or images they already have to the device, and for the system to receive them.

[1711] "Means for users to input instructions for their desired hairstyle and color" refers to a function that allows users to input their desired hairstyle and color via text or voice and transmit that information to the system.

[1712] "Means for processing an image based on the received image in accordance with the user's instructions using generative artificial intelligence" refers to a function that uses artificial intelligence technology to process the image based on the received image data and the user's instructions.

[1713] "Means of recognizing emotions by displaying a processed image to the user and analyzing the user's facial expressions and voice" refers to a function that displays a processed image to the user, analyzes the user's facial expressions and voice to understand their emotions, and reflects them in the system.

[1714] The "means for receiving a fine adjustment instruction from the user and processing the image again" is a function for receiving a fine adjustment request from the user and processing the image again based on the request.

[1715] The "means for saving the final processed image and sending it to an expert" is a function for saving the final processed image that the user is satisfied with and transferring it to an expert.

[1716] The present invention provides a system that allows a user to simulate hairstyles and colors in an autonomous vehicle and transmit the results to an expert. This system is realized using the following means.

[1717] User Actions

[1718] Using the interface inside the autonomous vehicle, users can take a photo of themselves or select a photo they have already taken from their gallery, then upload the selected photo to a dedicated application, which then sends the photo to a server.

[1719] Server Processing

[1720] The server receives the photo sent from the device and temporarily stores it. It provides an interface for the user to input instructions for the desired hairstyle and color. The user's instructions are sent to the server via the device, and the server analyzes them and provides them to the generative AI model. The generative AI model generates an image with the hairstyle and color applied based on the user's specifications.

[1721] Image generation and display

[1722] The image processed by the generative AI model is sent from the server to the device and displayed for the user to review. As the user reviews the image, the emotion engine analyzes their facial expressions and voice to recognize their emotions. If the user is not satisfied, it will suggest an alternative. Furthermore, it can accept instructions for fine-tuning and process the image again. This process is repeated until the user is finally satisfied.

[1723] Final confirmation and submission

[1724] When the user finally taps the "Decide" button on the image they are satisfied with, the device prepares to send the final edited image to the expert. The server forwards the edited image to the expert's designated contact and obtains the expert's confirmation of receipt. The results of the user's sentiment analysis are also sent to the expert, allowing the expert to understand the user's mood and preferences before providing the service.

[1725] Hardware and software used

[1726] Hardware: Camera, large display, smartphone, head-mounted display

[1727] Software: OpenCV (face detection and image processing), dlib (face detection), Keras (emotion recognition model), generative AI model

[1728] As a concrete example, a user can input instructions through an in-car interface based on a selfie they have uploaded, such as "I want shoulder-length hair, swept-back bangs, and an ash gray color." The server analyzes these instructions and processes the image using a generative AI model. When the user reviews the image, the emotion engine analyzes the user's facial expressions and automatically suggests "making the bangs a little shorter." The user then agrees to the reprocessing, and the server processes the image again using the generative AI model. This process is repeated, and the final image that the user is satisfied with is sent to the expert. The expert then confirms receipt and provides the service, taking into account the results of the emotion analysis.

[1729] Prompt Sentence Examples

[1730] "Long hair, blonde."

[1731] "Shoulder-length hair, swept-back bangs, ash gray."

[1732] As described above, the present invention enables simulation of hairstyles and colors that take into account the user's emotions within an autonomous vehicle, making it possible to smoothly communicate information to experts.

[1733] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1734] Step 1:

[1735] The user launches a dedicated application in the self-driving vehicle and takes a photo of their face or selects a photo they have already saved.

[1736] Input: A photo of your face taken or selected by the user

[1737] Output: Photo data

[1738] How it works: Take a photo using your smartphone or car camera, or select a photo from your gallery and upload it to the dedicated application.

[1739] Step 2:

[1740] The terminal transmits the uploaded photo data to the server.

[1741] Input: Photo data

[1742] Output: Photo data sent to the server

[1743] How it works: Your device uploads photo data to a server via your internet connection.

[1744] Step 3:

[1745] The server temporarily stores the received photo data and provides an interface for the user to input instructions for the desired hairstyle and color.

[1746] Input: Photo data

[1747] Output: Instruction input interface

[1748] How it works: The server stores the photo data in a database and prompts the user for input for the next processing step.

[1749] Step 4:

[1750] The user inputs the desired hairstyle and color and sends the instructions to the server.

[1751] Input: Hair style and color instructions (prompt text)

[1752] Output: User instructions

[1753] How it works: The user enters their desired hairstyle and color via text or voice into a field within the application and submits that information.

[1754] Step 5:

[1755] The server analyzes the user's instructions and supplies them to the generative AI model, which then generates an image with the hairstyle and color applied based on the received photo data and the user's instructions.

[1756] Input: Photo data, user instructions

[1757] Output: Processed image

[1758] How it works: The server analyzes the text of the user's instructions and passes them to a generative AI model to process the image.

[1759] Step 6:

[1760] The server transmits the generated processed image to the terminal and displays it to the user.

[1761] Input: processed image

[1762] Output: Image sent to user device

[1763] How it works: The server sends the generated image data to the user's device, which then displays the image.

[1764] Step 7:

[1765] The user can review the edited image and input instructions for fine-tuning, while the emotion engine analyzes the user's facial expressions and voice to recognize their emotions.

[1766] Input: User emotion data, fine-tuning instructions

[1767] Output: User emotion data and fine-tuning content

[1768] How it works: The application captures the user's facial expressions and voice using a camera and microphone, which the emotion engine analyzes. The user then inputs fine-tuning instructions.

[1769] Step 8:

[1770] The server receives the user's fine-tuning instructions and emotion data and requests the generative AI model to process it again.

[1771] Input: Fine-tuning instructions, user emotion data

[1772] Output: Reprocessed image

[1773] How it works: The server analyzes the emotion data and fine-tuning instructions, then passes new instructions to the generative AI model for reprocessing.

[1774] Step 9:

[1775] When the user finally taps the "Decide" button on the image they are satisfied with, the device prepares the final edited image for sending to the expert and notifies the server.

[1776] Input: Final processed image

[1777] Output: Prepared for sending to a specialist

[1778] How it works: The user taps the "OK" button, and the device notifies the server of the final processed image.

[1779] Step 10:

[1780] The server forwards the final processed image to the expert's designated contact and obtains the expert's acknowledgement of receipt. It also sends the user's emotion data to the expert.

[1781] Input: Final processed image, user emotion data

[1782] Output: Send to expert, acknowledge receipt

[1783] How it works: The server sends the final processed image and emotion data to the expert, receives confirmation data, and provides feedback to the user.

[1784] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the control target 443 to output the result of the specific processing. The microphone 238 acquires voice indicating a user input regarding the result of the specific processing. The control unit 46A transmits voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[1785] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1786] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the robot 414.

[1787] The emotion identification model 59 as an emotion engine may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to an emotion map (see FIG. 9), which is a specific mapping. Similarly, the emotion identification model 59 may determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[1788] FIG. 9 is a diagram illustrating an emotion map 400 on which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. Emotions closer to the center of the concentric circles are more primitive. Emotions representing states and actions arising from a state of mind are arranged on the outer edges of the concentric circles. The concept of emotion includes both affect and mental states. Emotions generally generated from reactions occurring in the brain are arranged on the left side of the concentric circles. Emotions generally induced by situational judgment are arranged on the right side of the concentric circles. Emotions generally generated from reactions occurring in the brain and induced by situational judgment are arranged on the upper and lower sides of the concentric circles. Furthermore, the emotion of "pleasure" is arranged on the upper side of the concentric circles, and the emotion of "discomfort" is arranged on the lower side. In this way, in the emotion map 400, multiple emotions are mapped based on the structure by which emotions are generated, and emotions that tend to occur simultaneously are mapped close to each other.

[1789] These emotions are distributed in the 3 o'clock direction on emotion map 400, and typically fluctuate between relief and anxiety. In the right half of emotion map 400, situational awareness dominates over internal sensations, resulting in a sense of calm.

[1790] The inside of emotion map 400 represents what is going on in the mind, and the outside of emotion map 400 represents behavior, so the further you go outside emotion map 400, the more visible the emotions become (the more they are expressed in behavior).

[1791] Human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. Emotions can also be created for robots, automobiles, and motorcycles, based on various balances, such as posture and remaining battery life. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. An emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on Voice Emotion Recognition and Emotional Brain Physiological Signal Analysis Systems, Tokushima University, Doctoral Dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map lists emotions belonging to the "reaction" domain, where sensation is dominant. The right half of the emotion map lists emotions belonging to the "situation" domain, where situational awareness is dominant.

[1792] The emotion map defines two emotions that promote learning. One is a negative emotion on the situation side, around the middle of "repentance" or "reflection." In other words, this occurs when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is a positive emotion on the response side, around "desire." In other words, this occurs when the robot experiences positive feelings such as "I want more" or "I want to know more."

[1793] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​indicating each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple pieces of training data that are combinations of user input and emotion values ​​indicating each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions that are located close to each other have similar values, as in the emotion map 900 shown in FIG. 10. FIG. 10 shows an example in which multiple emotions, "relieved," "calm," and "reassuring," have similar emotion values.

[1794] The system according to the present disclosure has been described above mainly with respect to the functions of the data processing device 12, but the system according to the present disclosure is not necessarily implemented on a server. The system according to the present disclosure may be implemented as a general information processing system. The present disclosure may be implemented, for example, as a software program running on a personal computer or an application running on a smartphone, etc. The method according to the present disclosure may be provided to users in the form of SaaS (Software as a Service).

[1795] In the above embodiment, an example was given in which the specific processing is performed by one computer 22, but the technology of the present disclosure is not limited to this, and the specific processing may be distributed and performed by a plurality of computers including the computer 22. For example, the data generation model 58 may be provided in an external device of the data processing device 12, and data may be generated in the external device in accordance with input data.

[1796] In the above embodiment, an example in which the specific processing program 56 is stored in the storage 32 has been described, but the technology of the present disclosure is not limited to this. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-transitory storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-transitory storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes the specific processing in accordance with the specific processing program 56.

[1797] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[1798] It is not necessary to store all of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store all of the specific processing program 56 in the storage 32; only a portion of the specific processing program 56 may be stored.

[1799] The hardware resource for executing a specific process can be any of the following types of processors: A processor, for example, is a CPU, which is a general-purpose processor that functions as a hardware resource for executing a specific process by executing software, i.e., a program. Another processor is a dedicated electrical circuit, such as an FPGA (Field-Programmable Gate Array), a PLD (Programmable Logic Device), or an ASIC (Application Specific Integrated Circuit), which is a processor with a circuit configuration designed specifically for executing a specific process. Each processor has built-in or connected memory, and each processor uses the memory to execute the specific process.

[1800] The hardware resource that executes the specific processing may be configured with one of these various processors, or may be configured with a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Also, the hardware resource that executes the specific processing may be a single processor.

[1801] As an example of a system configured with a single processor, first, one processor is configured by combining one or more CPUs and software, and this processor functions as a hardware resource that executes a specific process. Second, there is a system that uses a processor that realizes the functions of an entire system including multiple hardware resources that execute a specific process on a single IC chip, as typified by SoC (System-on-a-chip). In this way, a specific process is realized using one or more of the above-mentioned various processors as hardware resources.

[1802] Furthermore, the hardware structure of these various processors can be, more specifically, an electric circuit that combines circuit elements such as semiconductor devices. The specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps may be deleted, new steps may be added, or the processing order may be rearranged, without departing from the spirit of the invention.

[1803] The above-described description and illustrations are a detailed explanation of the parts related to the technology of the present disclosure and are merely an example of the technology of the present disclosure. For example, the above description of the configuration, functions, actions, and effects is an explanation of an example of the configuration, functions, actions, and effects of the parts related to the technology of the present disclosure. Therefore, it goes without saying that unnecessary parts may be deleted, new elements may be added, or replacements may be made to the above-described description and illustrations within the scope of the gist of the technology of the present disclosure. Furthermore, to avoid confusion and facilitate understanding of the parts related to the technology of the present disclosure, the above-described description and illustrations omit explanations of common technical knowledge that do not require particular explanation to enable the implementation of the technology of the present disclosure.

[1804] All publications, patent applications, and technical standards mentioned in this specification are herein incorporated by reference to the same extent as if each individual publication, patent application, or technical standard was specifically and individually indicated to be incorporated by reference.

[1805] The following is further disclosed regarding the above embodiment.

[1806] (Claim 1)

[1807] means for receiving a user-captured or selected image;

[1808] a means for the user to input desired hairstyle and color instructions;

[1809] A means for processing an image based on the received image in accordance with instructions from the user using a generating artificial intelligence;

[1810] a means for displaying the processed image to the user, receiving fine adjustment instructions from the user, and processing the image again;

[1811] A means for saving and sending the final processed image to a hairdresser;

[1812] A system including:

[1813] (Claim 2)

[1814] A means for the generation artificial intelligence to analyze the received image and generate an image of the optimal hairstyle and color based on the user's instructions;

[1815] A means for providing an interface for a user to check the generated image and give instructions for reprocessing;

[1816] The system of claim 1 further comprising:

[1817] (Claim 3)

[1818] means for transmitting the generated final hairstyle image to a hairdresser, obtaining a hairdresser's acknowledgement of receipt, and providing feedback to the user;

[1819] A means for storing transmission history;

[1820] The system of claim 1 further comprising:

[1821] "Example 1"

[1822] (Claim 1)

[1823] means for receiving a user-captured or selected image;

[1824] a means for the user to input desired hairstyle and color instructions;

[1825] A means for processing an image based on the received image in accordance with instructions from the user using a generating artificial intelligence;

[1826] a means for displaying the processed image to the user, receiving fine adjustment instructions from the user, and processing the image again;

[1827] A means for saving and sending the final processed image to a hairdresser;

[1828] A system including:

[1829] (Claim 2)

[1830] A means for the generation artificial intelligence to analyze the received image and generate an image of the optimal hairstyle and color based on the user's instructions;

[1831] A means for providing an interface for a user to check the generated image and give instructions for reprocessing;

[1832] The system of claim 1 further comprising:

[1833] (Claim 3)

[1834] means for transmitting the generated final hairstyle image to a hairdresser, obtaining a hairdresser's acknowledgement of receipt, and providing feedback to the user;

[1835] A means for storing transmission history;

[1836] The system of claim 1 further comprising:

[1837] "Application Example 1"

[1838] (Claim 1)

[1839] means for receiving a user-captured or selected image;

[1840] a means for the user to input desired hairstyle and color instructions;

[1841] A means for processing an image based on the received image in accordance with user instructions using a generative AI model;

[1842] a means for displaying the processed image to the user, receiving fine adjustment instructions from the user, and processing the image again;

[1843] means for storing and transmitting the final processed image to a professional;

[1844] a means for managing the user's reservations;

[1845] A system including:

[1846] (Claim 2)

[1847] A means for the generative AI model to analyze the received image and generate an image of the optimal hairstyle and color based on the user's instructions;

[1848] A means for providing an interface for a user to check the generated image and give instructions for multiple reprocessing;

[1849] A means for improving the accuracy of instructions to the generative AI model by prompt sentences;

[1850] The system of claim 1 further comprising:

[1851] (Claim 3)

[1852] means for transmitting the generated final hairstyle image to a professional, obtaining a professional acknowledgement of receipt, and providing feedback to the user;

[1853] A means for storing transmission history;

[1854] A means of providing professional staff with data to help ensure smooth service when users visit stores;

[1855] The system of claim 1 further comprising:

[1856] "Example 2: Combining Emotion Engines"

[1857] (Claim 1)

[1858] means for receiving a user-captured or selected image;

[1859] a means for the user to input desired hairstyle and color instructions;

[1860] A means for processing an image based on the received image in accordance with a user's instructions using a generative artificial intelligence model;

[1861] a means for displaying the processed image to the user, receiving fine adjustment instructions from the user, and processing the image again;

[1862] A means for saving and sending the final processed image to a hairdresser;

[1863] A means of recognizing user emotions and automatically making emotion-based suggestions;

[1864] A system including:

[1865] (Claim 2)

[1866] A means for the generation artificial intelligence model to analyze the received image and generate an image of the optimal hairstyle and color based on the user's instructions;

[1867] A means for providing an interface for a user to check the generated image and give instructions for reprocessing;

[1868] A means for making automatic suggestions based on a user's emotions using an engine that analyzes the user's emotions;

[1869] The system of claim 1 further comprising:

[1870] (Claim 3)

[1871] means for transmitting the generated final hairstyle image to a hairdresser, obtaining a hairdresser's acknowledgement of receipt, and providing feedback to the user;

[1872] A means for storing transmission history;

[1873] A means for providing the user's sentiment analysis results to a hairdresser;

[1874] The system of claim 1 further comprising:

[1875] "Application example 2 when combining emotion engines"

[1876] (Claim 1)

[1877] means for receiving a user-captured or selected image;

[1878] a means for the user to input desired hairstyle and color instructions;

[1879] A means for processing an image based on the received image in accordance with instructions from the user using a generating artificial intelligence;

[1880] A means for displaying the processed image to the user and recognizing emotions by analyzing the user's facial expressions and voice;

[1881] A means for receiving fine adjustment instructions from the user and reprocessing the data;

[1882] A means to save and send the final processed image to a professional;

[1883] A system including:

[1884] (Claim 2)

[1885] A means for the generation artificial intelligence to analyze the received image and generate an image of the optimal hairstyle and color based on the user's instructions;

[1886] A means for providing an interface that allows a user to check the generated image, recognize emotions by analyzing the user's facial expressions and voice, and give instructions for reprocessing;

[1887] The system of claim 1 further comprising:

[1888] (Claim 3)

[1889] means for sending the generated final hairstyle image to the expert, obtaining the expert's acknowledgement of receipt, and providing feedback to the user;

[1890] A means for storing transmission history;

[1891] The system of claim 1 further comprising: [Explanation of symbols]

[1892] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Device 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robot< / url:> < / url:> < / url:> < / url:>

Claims

1. means for receiving a user-captured or selected image; a means for the user to input desired hairstyle and color instructions; A means for processing an image based on the received image in accordance with instructions from the user using a generating artificial intelligence; a means for displaying the processed image to the user, receiving fine adjustment instructions from the user, and processing the image again; A means for saving and sending the final processed image to a hairdresser; A system including:

2. A means for the generation artificial intelligence to analyze the received image and generate an image of the optimal hairstyle and color based on the user's instructions; A means for providing an interface for a user to check the generated image and give instructions for reprocessing; The system of claim 1 further comprising:

3. means for transmitting the generated final hairstyle image to a hairdresser, obtaining a hairdresser's acknowledgement of receipt, and providing feedback to the user; A means for storing transmission history; The system of claim 1 further comprising:

Images